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Preface 

This  research  was  conducted  to  shed  some  light  on  the  Air  Force’s 
Acquisition  Strategy  Panel  process.  From  the  research,  I  found  that  a 
lot  of  the  early  grunt) lings  about  the  ASP  had  given  way  to  the  feeling 
that  the  ASP  was  something  the  Air  Force  was  doing  right.  This  success 
can  be  traced  to  the  program  managers  and  panel  members  involved  with 
the  process  and  their  genuine  desire  to  "do  it  right  the  first  time."  I 
hope  that  this  research  will  serve  as  an  example  for  other  government 
agencies  involved  in  complex  acquisitions.  A  glossary  of  acronyms  used 
throughout  this  document  is  included  as  Appendix  G. 

A  thesis  is  never  written  in  a  vaccuum.  Accordingly,  their  are 
many  people  to  whom  I  am  indebted.  The  first  is  my  thesis  advisor, 

LtCol  Curtis  Cook,  without  whose  assistance  this  document  would  not  have 
been  possible.  The  Headquarters  AFSC  ASP  Secretariat,  headed  by  Ms 
Lilian  Stone,  was  of  invaluable  assistance  in  identifying  the  program 
offices  which  had  been  through  the  process  and  the  documentation  related 
to  the  ASP  process  which  were  considered  in  the  research.  I  would  also 
like  to  thank  Mr  Ed  Martin  and  Mr  Jim  Witham  of  the  Aeronautical  Systems 
Division  ASP  Secretariat,  and  LtCol  Pam  Casey  and  her  staff  of  the 
Electronic  Systems  Division  ASP  Secretariat.  And  I  would  like  to 
express  my  sincere  thanks  to  the  men  and  women  who  took  time  out  of 
their  schedules  to  participate  in  the  formal  interviews. 

And  finally,  I  would  like  to  thank  a  fine  young  man,  Mr  C.J. 

Davis,  and  the  Big  Brothers  and  Big  Sisters  of  Greater  Dayton  for  giving 
me  something  to  believe  in  during  my  stay  at  AFIT. 

Scott  C.  Hardiman 
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ABSTRACT 

The  Acquisition  Strategy  Panel  (ASP)  was  implemented  by  Air  Force 
Systems  Command  (AFSC)  to  assist  program  managers  developing  strategies 
to  acquire  new  weapon  systems.  When  it  was  first  implemented,  the 
perception  of  many  program  managers  was  that  it  was  just  another 
bureaucratic  exercise.  Based  on  this  perception,  and  amidst  calls  for 
streamlining  the  acquisition  process,  a  thorough  examination  of  the 
ASP’s  value  was  in  order.  To  this  end,  29  program  managers  and  panel 
members  with  first-hand  knowledge  of  the  ASP  process  were  interviewed. 
Five-point  Likert  opinion  scales  were  used  throughout  the  structured 
interviews.  The  data  refuted  the  early  impressions  of  the  ASP  by 
revealing  that  its  recommendations  were  being  implemented,  that  these 
recommendations  were  having  positive  impacts  on  the  programs  involved, 
and  that  the  ASP  was  considered  a  valuable  tool  by  the  majority  of  the 
respondents.  However,  this  value  was  primarily  of  a  qualitative  nature. 
The  research  also  revealed  several  areas  in  the  process  which  could  be 
improved.  These  included  integrating  the  ASP  into  the  Pregram  Executive 
Officer  structure  and  delegating  responsibility  for  production  ASPs  to 
lower  levels.  A  total  of  14  recommendations  were  made  which  have  the 
potential  to  make  a  good  process  better. 
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THE  IMPACT  OF  THE  ACQUISITION  STRATEGY  PANEL  ON  PROGRAM  EFFECTIVENESS 


I .  Introduction 

One  of  the  first  steps  in  the  development  and  procurement  of  a  new 
weapon  system,  once  the  need  for  such  a  system  has  been  established,  is 
the  development  of  an  acquisition  strategy.  The  strategy  provides  an 
organized  and  consistent  approach  to  meeting  program  objectives  within 
known  constraints.  A  sound  strategy  minimizes  technical,  schedule,  and 
cost  risks  in  order  to  achieve  program  stability.  The  acquisition 
strategy  develops  technological  options,  explores  design  concepts,  and 
plans  and  conducts  acquisition  activities  to  attain  balanced  cost 
effectiveness.  Because  of  its  importance  to  the  overall  program,  it 
must  be  developed  as  early  in  the  program  life  cycle  as  possible. 

(27:3-1) 

Air  Force  Systems  Command  (AFSC),  the  major  command  charged  with 
procuring  new  weapon  systems  for  the  US  Air  Force,  implemented  a  process 
known  as  the  Acquisition  Strategy  Panel  (ASP)  to  assist  the  program 
manager  in  the  development  of  the  program’s  acquisition  strategy.  The 
purpose  of  the  ASP  was  to  bring  as  much  expertise  and  experience  as 
possible  to  bear  in  planning  the  acquisition  strategy.  The  total 
program  was  examined  by  experts  in  all  of  the  functional  fields  of 
acquisition  to  insure  that  the  strategy  was  workable,  addressed  risk, 
and  was  appropriate  for  the  type  of  work  to  be  performed. 

General  Issue 

As  conceived,  the  ASP  served  as  a  valuable  program  management  tool 

by  ensuring  that  all  relevant  issues  that  had  the  potential  to  affect  a 
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program  were  explored.  In  practice  however,  some  critics  contended  that 
the  ASP  process  had  become  just  another  bureaucratic  square-filling 
exercise  which  the  program  manager  had  to  endure  on  the  way  to  fielding 
a  weapon  system.  The  truth  lies  somewhere  bp tween  these  two  extremes. 
The  purpose  of  this  thesis  was  to  determine  if  the  ASP  was  living  up  to 
its  initial  goals.  With  the  demand  for  streamlined  procurement 
practices  growing  every  day,  and  with  realignment  of  the  procurement 
hierarchy  underway,  the  contribution  of  the  ASP  to  program  success  is 
subject  to  scrutiny. 

Problem  Statement 

With  the  goal  of  determining  whether  or  not  the  ASP  should  be 
continued  or  improved,  the  following  specific  question  had  to  be 
answered.  What  impact  did  the  ASP  have  on  program  effectiveness?  If 
the  answer  to  this  question  showed  a  negative  or  marginal  impact, 
recommendations  for  improving  the  process  or  arguments  for 
discontinuance  of  the  ASP  altogether  would  be  advanced. 

Research  Objectives 

Preliminary  discussions  with  personnel  who  have  been  involved  with 
the  ASP  process  revealed  a  generally  negative  picture  of  the  usefulness 
of  AFSC  conducted  Acquisition  Strategy  Panels.  To  develop  a  more 
complete  and  accurate  portrayal  of  the  situation,  the  following 
investigative  questions  were  asked  of  personnel  'with  first-hand 
knowledge  of  the  process. 

1.  How  often  was  a  program  manager’s  strategy  redirected  by  the 
ASP?  What  was  the  overall  quality  of  the  recooroendations  received? 
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2.  Once  reconmendat i ons  had  been  received,  did  program  managers 
implement  them?  Did  the  process  or  implementation  of  the 
recommendations  have  an  effect  on  contract  award  schedule? 

3.  If  the  reconmendat ions  were  implemented  by  the  program 
manager,  did  they  result  in  positive,  negative,  or  neutral  impacts  to 
the  overall  success  of  the  program? 

4.  What  kinds  of  strategic  help  were  program  manager's  getting? 
What  kinds  of  strategic  help  did  program  managers  want  or  need? 

Scope  and  Limitations 

This  thesis  will  examined  the  impact  of  AFSC-conducted  ASPs  on 
program  effectiveness.  Because  of  a  lack  of  data  in  this  area,  the 
research  was  based  on  interviews  with  personnel  were  involved  in  the 
process.  In  practice,  the  ASP  was  conducted  at  three  levels.  For  the 
purpose  of  this  thesis  however,  only  those  personnel  involved  in  the 
highest  level  ASPs,  or  "joint  ASPs,"  were  interviewed.  A  joint  ASP  is 
one  which  is  co-chaired  by  Headquarters  AFSC  and  the  program’s  product 
division.  Personnel  involved  in  lower  level  ASPs  were  not  interviewed. 

Background 

The  1980 ’s  have  seen  tumultuous  change  in  the  way  that  the 
Department  of  Defense  (DoD)  and  its  component  services  buy  new  weapon 
systems  and  the  spare  parts  to  support  than.  These  changes  have  been 
brought  about  because  of  the  acquisition  horror  stories  of  the  early 
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80  *s  such  as  a  $9600  alien  wrench  for  the  F-16,  a  $7660  coffee  pot  for 
the  C-5  transport  that  could  survive  conditions  which  would  kill  the 
aircraft’s  flight  crew,  a  $110  diode  estimated  to  actually  cost  9  cents, 
a  $700  hanraer,  bolts  at  $17.59  each,  and  a  "toilet  seat"  for  $318 
(16:61).  With  these  stories,  it  is  no  wonder  that  there  have  been  calls 
for  changes  in  DoD  procurement  policy. 

Part  of  the  problem  may  lie  in  the  complexity  of  the  DoD 
procurement  structure.  The  defense  acquisition  system  is  an  extremely 
large  business  enterprise  involving  thousands  of  personnel,  billions  of 
dollars,  myriads  of  procedures  and  regulations,  and  the  intangible 
aspect  of  politics.  Management  oversight  in  any  form  contributes  to 
this  complexity.  Seme  of  this  oversight  is  necessary  and  contributes  to 
the  overall  success  of  the  program  while  some  does  not.  With  all  of  the 
calls  for  acquisition  streamlining  and  reductions  in  budgets  and 
manpower,  the  usefulness  of  the  additional  oversight  created  by  AFSC- 
conducted  ASPs  is  questionable. 

There  1ms  been  a  lot  of  emphasis  of  late  on  the  development  of 
acquisition  strategies  but  little  on  the  actual  execution  of  these 
strategies  (30:59).  In  his  thesis  on  acquisition  strategy  development, 
Bissett  recommended  that  additional  study  should  be  conducted  to  ".  .  . 
assess  the  impact  of  the  program  documentation  and  review  process  .  .  . 
on  the  strategy  options  available  to  the  program  manager"  (3:71).  This 
thesis  will  attempt  to  bridge  the  gap  between  the  planning  and  the 
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execution  of  strategies  by  focusing  on  acquisition  strategy  formulation 
via  AFSC’s  ASP  process  and  its  impact  on  program  effectiveness.  The 
next  chapter  will  probe  the  role  of  acquisition  strategy  planning  in 
program  management,  the  ASP  process  in  detail,  and  its  impact  in  view  of 
the  recent  reform  issues  surrounding  defense  procurement. 
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II.  Discussion  of  the  Literature 

This  chapter  examines  the  literature  concerning  the  role  of  the 
Air  Force  (AF)  acquisition  strategy  panel  process  in  the  acquisition  of 
major  AF  weapon  systems.  First,  a  general  description  of  the  Department 
of  Defense  (DoD)  major  system  acquisition  process  will  be  given  and 
significant  DoD  and  AF  guidance  in  this  area  will  be  highlighted.  Next, 
a  look  at  acquisition  strategy  development  and  execution  will  be  taken. 
This  will  be  followed  by  a  detailed  description  of  the  AF's  acquisition 
strategy  panel  (ASP)  process.  Finally,  major  initiatives  which  may 
affect  future  AF  acquisition  activities  will  be  examined. 

Overview  of  Defense  Acquisition 

To  understand  the  role  of  acquisition  strategy  in  the  development 
and  fielding  of  new  weapon  systems,  one  must  have  an  understanding  of 
the  defense  acquisition  process.  The  acquisition  of  new  weapon  systems 
for  the  nation’s  military  forces  has  been  taking  place  since  the 
founding  of  the  nation.  The  term  "acquisition"  itself,  however,  was 
only  adopted  by  DoD  in  the  1970 ’s  to  describe  the  total  process  of 
planning  and  management  required  to  meet  the  mission  needs  of  DoD 
(29:20).  According  to  the  Federal  Acquisition  Regulation  (FAR), 
acquisition  is  defined  as 

.  .  .  the  acquiring  by  contract  with  appropriated  funds  of 
supplies  or  services  by  and  for  the  use  of  the  Federal  Government 
through  purchase  or  lease,  whether  the  supplies  or  services  are 
already  in  existence  or  must  be  created,  developed,  demonstrated 
and  evaluated.  (33:Para  2.101) 

This  definition  implies  that  acquisition  is  an  extremely  large 

process.  Indeed,  DoD  acquisition  involves  billions  of  dollars  and 

thousands  of  personnel  each  year.  The  majority  of  this  business  is 

6 


conducted  through  large  contracts  with  private  firms.  Sherman  found 
that  the  DoD  typically  is  responsible  for  over  three  quarters  of  the 
federal  dollars  awarded  by  contract  each  year  (29:18).  Most  of  this 
money  was  spent  through  a  relatively  small  number  of  high  value 
contracts . 

Accordingly,  the  DoD  acquisition  process  centers  arouri  the 
awarding  and  monitoring  of  contracts  with  private  industry.  The  process 
can  be  broken  down  into  four  major  tasks  for  the  government's 
responsible  procuring  agency.  Hie  first  of  these  tasks  is  the 
development  of  an  acquisition  strategy.  This  topic  will  be  covered  in 
much  more  detail  in  the  next  two  sections  of  this  chapter.  Once  the 
strategy  has  been  developed,  a  solicitation  is  prepared  which  seeks 
sources  for  the  product  or  service  required  by  the  government. 

Following  the  solicitation,  the  source  is  selected  from  the  firms 
responding  with  proposals  to  fulfill  the  contract  and  the  contract  is 
awarded.  Upon  contract  award,  the  procuring  agency's  task  becomes  one 
of  administering  the  contract  and  monitoring  the  contractor’s 
performance.  Figure  1  illustrates  this  process.  These  tasks  take  place 
throughout  the  acquisition  life  cycle  of  the  system  or  service  being 
procured  and  are  guided  by  DoD  directives  and  AF  regulations.  (31:19- 
27) 

Current  DoD  and  Air  Force  Acquisition  Policy.  Major  DoD 
acquisition  policies  are  set  forth  primarily  in  DoD  Directive  (DoDD) 
5000.1,  Major  and  Non-Major  Defense  Acquisition  Programs,  and  DoD 
Instruction  fDoDI)  5000.2,  Defense  Acquisition  Program  Procedures.  Air 
Force  acquisition  policy  is  established  in  Air  Force  Regulation  (AER) 
800-2,  Acquisition  Program  Management  (along  with  Air  Force  Systems 
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Figure  1.  Thybony  Acquisition  Process  Model 

Command  Supplement  1  to  AFP  800-2 ) .  These  documents  have  their  roots  in 
the  Federal  Acquisition  Regulation  (FAR)*  implemented  in  April  1984, 
which  guides  all  federal  procurement  actions.  Numerous  other  directives 
and  regulations  exist  which  establish  more  detailed  policies.  This 
review  will  examine  only  the  major  policy  directives  and  regulations. 

The  major  DoD  directives  related  to  acquisition  policy  include 
many  of  the  recommendations  of  the  President's  Blue  Ribbon  Conmission  on 
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Defense  Management  (the  Packard  Commission) . 1  DoDD  5000.1  sets  forth 
the  major  policies  for  defense  acquisitions  while  DoDI  5000.2 
establishes  specific  procedures. 

The  Packard  Commission  Report  called  for  the  streamlining  oi  the 
defense  acquisition  bureaucracy  and  the  elimination  of  wasteful 
processes  where  possible,  among  other  things.  DoDD  5000.1  streamlines 
the  acquisition  chain  of  command  for  major  programs2  by  imposing  only 
two  management  levels  between  the  program  manager  and  the  Defense 
Acquisition  Executive  (DAE).  The  DAE  serves  as  " .  .  .  the  principal 
advisor  to  the  Secretary  of  Defense  on  all  matters  pertaining  to  .  . 

DoD  acquisition  programs.  Between  the  program  manager  and  the  DAE  are 
the  Service  Acquisition  Executive  (SAE) ,  who  serves  as  the  top 
acquisition  official  in  each  military  branch,  and  the  Program  Executive 
Officer  (PEJO) ,  who  is  responsible  for  administering  several  major  and 
non-major  acquisition  programs  within  the  military  branch.  A  program 
manager  (FM)  is  responsible  for  managing  a  specific  program  within  a 
military  branch.  The  directive  also  calls  on  all  military  branches  to 
streamline  their  acquisition  bureaucracies  for  both  major  and  non-major 
programs.  It  encourages  only  one  management  tier  between  the  program 
manager  and  the  SAE  for  those  programs  which  do  not  include  DAE 
oversight.  (10:2-3) 


1The  Packard  Commission  was  chartered  in  1986  by  the  Reagan  Presidential 
Administration  to  examine  and  recomaend  changes  to  the  DoD  acquisition  process 
(6:30). 

2A  DoD  major  program  is  generally  defined  as  a  program  that  will  require  at 
least  $200  million  in  research  and  development  funding  and/or  a  total  of  at  least 
$1  billion  in  procurement  funding,  or  other  programs  as  designated  by  the 
Secretary  of  Defense  (8:2). 
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For  the  Air  Force,  this  has  meant  removing  Air  Force  Systems 
Command  (AFSC)  from  the  program  management  chain  of  command.  The  PEOs 
for  the  Air  Force  will  be  assigned  to  Headquarters  USAF  while  the  SAE 
will  reside  in  the  office  of  the  Secretary  of  the  Air  Force.  AFSC  will 
be  concerned  primarily  with  providing  the  infrastructure  and  resources 
the  program  managers  need  to  adequately  manage  their  programs.  Much  of 
this  change  is  just  starting  to  take  place  as  of  the  writing  of  this 
thesis.  It  remains  to  be  seen  how  the  details  will  be  worked  out.  This 
does  not,  however,  change  the  acquisition  life  cycle  of  a  major  system. 

Major  System  Acquisition  Cycle.  The  current  acquisition  cycle 
model  is  described  in  DoDD  5000.1.  It  was  developed  in  1976  by  the 
Office  of  Federal  Procurement  Policy  (OFPP)  and  was  implemented  DoD- 
wide  with  the  publication  of  Office  of  Management  and  Budget  Circular  A- 
109  that  same  year.  The  OFPP  model  was  based  upon  input  received  from 
the  Commission  on  Government  Procurement  in  1972.  (29:224) 

Figure  2  illustrates  this  cycle.  The  process  consists  of  five 
distinct  phases  separated  by  milestone  decision  points .  it  begins  with 
the  identification  of  a  sjpecific  need  for  a  new  system.  Once  the  need 
has  been  established,  the  concepts  for  meeting  the  need  are  explored  and 
defined.  The  more  promising  concepts  are  then  demonstrated  and 
validated  to  determine  feasibility.  Feasible  concepts  progress  to  full 
scale  development  in  preparation  for  production.  Next,  the  developed 
concepts  are  produced,  followed  by  fielding,  and  in-service  support. 
Finally  determination  for  retirement,  modification  or  replacement  of  the 
system  is  made.  These  phases  are  tied  together  through  continuous 
analysis  of  evolving  mission  requirements.  Mission  analysis  is  used  to 
identify  deficiencies  in  a  mission  area  and  to  find  more  effective  ways 
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to  perform  that  mission.  The  milestone  decision  points,  along  with 
criteria  established  for  each  point  in  DoDI  5000.2,  are  used  by  the 
Defense  Acquisition  Board  (DAB)  to  make  a  recommendation  to  the  Defense 
Acquisition  Executive  (DAE)  as  to  whether  or  not  the  program  should 
proceed  into  the  next  phase  of  the  acquisition.  It  should  be  noted  that 
at  the  time  of  this  writing  the  DoD  acquisition  life-cycle  was  being 
revised  to  eliminate  milestone  V.  This  revision,  however  does  not 
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Figure  2.  DoD  Major  System  Acquisition  Lifecycle 


This  model  was  designed  to  get  top  level  attention  on  mission  need 
and  goal  determination;  to  integrate  the  needs  identification, 
budgeting,  contracting  and  management  processes;  to  get  early  direction 
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for  research  and  development  (R&D)  efforts;  to  improve  private  sector 
innovation  in  meeting  the  nation’s  needs;  to  avoid  premature  coonitment 
to  full  scale  development  (FSD)  and  production;  and  to  provide  for  the 
early  communication  with  Congress  by  relating  needs  to  acquisition. 
(29:224) 

Hie  way  these  life  cycle  phases  are  conducted  for  each  system  . 
.  delineates  a  unique  and  complex  acquisition  strategy"  (29:224).  The 
role  of  sound  acquisition  strategy  development  is  discussed  in  the  next 
section. 

Developing  Strategies  for  the  Acquisition  of  Major  Systems 

The  word  "strategy"  implies  long  range  planning  taking  all 
possible  scenarios  into  account.  Armies  use  strategies  in  planning  wars 
and  battles;  businesses  use  strategies  in  planning  long  term  investments 
and  product  lines.  And  so  too  must  the  program  manager  use  strategies 
to  plan  for  the  acquisition  of  modem  weapon  systems.  These  program 
management  strategies  are  known  as  "acquisition  strategies"  in  DoD 
procurement . 

What  is  an  Acquisition  Strategy?  The  term  "acquisition  strategy" 
itself  means  different  things  to  different  people.  Indeed,  the 
Acquisition  Strategy  Guide  published  by  the  Defense  Systems  Management 
College  states  that  "there  is  no  cotnnon  working  definition  of 
acquisition  strategy  or  any  consistent  agreement  on  its  structure  and 
composition,  nor  is  there  comprehensive  guidance  on  how  to  proceed  in 
developing  and  executing  an  acquisition  strategy"  (27:1-1).  This  lack 
of  a  common  definition  causes  ”...  part  of  the  problem  of 
understanding  acquisition  strategy"  among  defense  procurement 
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specialists  (34:10).  In  1976,  0MB  Circular  A-109  defined  strategy 
formulation  as  the  "...  initial  task  of  a  program  manager  and  as  a 
task  that  requires  thinking  through  all  factors  relating  to  the  program 
and  treating  each  so  that  all  program  objectives  will  be  met"  (29:227). 
The  FAR  defines  it  as  the  "program  manager’s  overall  plan  for  satisfying 
the  mission  need  in  the  most  effective,  economical ,  and  timely  manner" 
(33  :para  34.004).  The  DoD  defined  it  in  1980  as  "the  conceptual  basis 
of  the  overall  plan  that  a  program  manager  follows  in  program  execution" 
(27:1-1).  A  broad  definition  has  also  been  suggested  by  Swanson  who 
states  that  "the  term  'acquisition  strategy’  is  simply  a  more  formal 
name  given  to  the  process  that  every  individual  .  .  .  uses  when  deciding 
how  to  convert  available  resources  into  a  desired  product  or  service" 
(30:59).  It  has  also  been  referred  to  as  the  total  roadmap  which  guides 
the  program  manager  in  the  execution  of  a  program  (34:11).  This  last 
statement  will  serve  as  the  working  definition  of  acquisition  strategy 
for  this  thesis. 

At  this  point,  it  is  important  to  note  the  difference  between  an 
"acquisition  strategy"  and  a  "procurement  strategy."  The  distinction 
lies  in  their  respective  objectives.  A  procurement  strategy  is 
concerned  with  obtaining  an  optimum  source  selection  while  an 
acquisition  strategy  is  concerned  with  the  brooder  objective  of 
satisfying  mission  needs.  They  are  interdependent  processes  with 
procurement  strategy  being  a  subset  of  acquisition  strategy.  (29:216) 

While  the  definition  lacks  specificity,  there  is  no  lack  of 
regulatory  reference  on  the  subject.  DoDD  5000.1  requires  tailored 
acquisition  strategies  to  identify  trade-offs  between  cost  and 
performance,  to  identify  cost  drivers  and  producibility  factors,  and  to 
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develop  the  appropriate  business  and  technical  approaches  for  these 
issues  ( 10 : 5-6 ) .  DoDD  5000 . 2  states  that  the  acquisition  strategy  will 
be  addressed  in  the  System  Concept  Paper  at  Milestone  I,  which  is  the 
point  at  which  the  program  progresses  into  the  Demonstration/Validation 
development  phase  (9:7).  Guiding  Air  Force  procurement  activities  is 
APR  800-2,  which  requires  that  the  strategy  be  documented  in  the  Program 
Management  Plan  which  is  an  initial  planning  document  for  AF  programs 
(7:4). 

The  Purpose  of  Acquisition  Strategies.  With  such  a  broad 

definition,  the  acquisition  strategy  serves  as  a  road  map  of  the 

acquisition  process  for  a  program,  an  overall  approach  to  meet  broad 

program  objectives,  a  conceptual  framework  for  conducting  the 

acquisition  of  a  system,  and  as  an  integrating  mechanism  against  which 

functional  and  business  considerations  are  planned  (34:13).  The  FAR 

states  that  the  purpose  of  an  acquisition  strategy  is  to  provide  a 

.  .  .  process  by  which  the  efforts  of  all  personnel  responsible 
for  an  acquisition  are  coordinated  and  integrated  through  a 
comprehensive  plan  for  fulfilling  the  agency  need  in  a  timely 
manner  and  at  a  reasonable  cost.  It  includes  developing  the 
overall  strategy  for  managing  the  acquisition.  (33:para  7.101) 

The  acquisition  strategy  is  also  used  as  a  consensus  building 

tool.  It  provides  a  focus  for  the  program  team  and  serves  as  an 

agreement  between  the  program  manager  and  the  chain  of  coooand  on  how 

the  acquisition  will  be  conducted.  Thinking  through  the  total  program 

early  on  during  the  development  of  the  strategy  can  "bring  about  an 

objective  appraisal  in  an  effort  to  find  the  most  economical,  effective, 

and  efficient  manner  in  which  to  proceed"  (29:313).  Wickert,  in  his  Air 

Command  and  Staff  College  Student  Report,  echoes  this  thought  by  stating 

that  a  well  developed  strategy  "...  can  provide  the  basis  for 
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structuring  the  optimum  approach  to  conduct  the  business  of  acquiring 
new  weapon  systems"  (34:1). 

Facing  projected  cuts  xn  the  defense  budget  and  high  public 

interest  in  stories  of  mismanagement  of  programs  by  the  military 

services,  the  need  for  military  managers  to  practice  sound  management  is 

imperative.  A  program  that  starts  off  on  the  wrong  footing  will  face 

increasingly  large  obstacles.  The  way  to  optimize  management  of  a 

program  is  to  develop  a  sound  acquisition  strategy  early  on  and  update 

it  as  the  program  progresses  through  the  system  life  cycle.  Wickert 

found  that,  among  the  DoD  contractors  he  interviewed,  many  believed  that 

sound  strategies  on  the  part  of  the  government  were  key  to  the  success 

of  a  program.  Indeed,  not  developing  a  strategy  can  lead  to  crisis 

management  and  functional  discord  among  the  program  team.  In  an  era  of 

limited  acquisition  resources,  "a  sound  acquisition  strategy  is 

paramount  to  best  employ  {those}  limited  resources  and  solve  the 

problems  a  program  manager  faces."  (34:1-5) 

Requirements  for  an  Acquisition  Strategy.  To  be  effective,  an 

acquisition  strategy  must  contain  certain  elements.  These  elements 

encompass  a  broad  range  of  issues.  In  his  book  on  government 

procurement,  Sherman  states  that 

.  .  .  strategic  matters  require  integrative  thinking  that 
encompasses  market  trends,  economic  forces,  public  policy 
developments,  source  availabilities,  source  development, 
competition,  and  productivity  issues,  technological  change, 
ethical  concerns,  economy  and  efficiency  of  the  process,  and  the 
general  problem  of  information  flow  across  organizational 
boundaries .  (29:5) 

Thybony  breaks  down  acquisition  strategy  development  and  planning 
into  four  general  areas  with  the  ultimate  goal  of  providing  an  advance 
comprehensive  plan  developed  to  fill  an  agency  need  via  contract.  These 
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general  areas  are  the  determination  of  government  requirements,  the 
development  of  a  requirement  specification,  the  preparation  of  a 
purchase  request,  and  documenting  the  strategy.  Depending  on  the  phase 
of  the  program,  some  or  all  of  the  items  listed  are  involved  during 
strategy  development.  (31:19-22) 

Determination  of  Government  Requirements.  Hie  determination 
of  government  requirements  consists  of  activities  such  as  mission 
analysis,  development  of  a  need  statement ,  the  establishment  of  a 
contracting  office  liaison,  starting  procurement  planning,  formulating 
and  approving  the  program,  preparing  a  preliminary  cost  estimate, 
preparing  the  budget  authorization  and  appropriation,  and  selecting  and 
approving  the  project.  (31:19-20) 

Requirement  Specification.  Development  of  the  requirement 
specification  involves  establishing  the  program  management  team, 
developing  the  statement  of  work,  and  developing  the  functional, 
performance  and  design  specifications.  In  addition,  a  list  of  required 
federal  and  military  specifications  is  prepared,  quality  and  quantity 
requirements  are  established,  and  contractor  delivery  and  performance 
requirements  are  established.  Other  requirements  established  at  this 
time  include  special  requirements  fcr  financial  reporting, 
subcontracting,  technical  data,  transportation,  government  furnished 
property,  spares  provisioning  and  industrial  security.  (31:20) 

Purchase  Request.  The  purchase  request  will  include 
information  on  all  of  the  acquisition  requirements,  the  source  list  or 
sole  source  justification,  proposal  evaluation  and  source  selection 
criteria,  contract  cost  estimates,  and  the  citation  of  funds  to  be 
committed.  (31:20) 


Documentation .  Documentation  of  the  acquisition  strategy 
occurs  in  several  reports .  Initial  strategy  is  included  in  the  original 
justification  for  the  program  while  the  program  manager's  overall 
strategy  for  the  entire  life-cycle  of  the  program  is  contained  in  the 
Program  Management  Plan  (PMP).  However,  the  strategy  for  the  current 
acquisition  phase  of  the  program  is  found  in  two  important  documents. 
These  documents  are  the  Acquisition  Plan,  prepared  by  the  program 
manager,  and  the  Procurement  Plan,  prepared  by  the  contracting  officer. 
(31:21,97) 

The  Acquisition  Plan.  The  document  that  discusses  the 
program’s  overall  acquisition  strategy  for  the  current  acquisition  phase 
is  the  Acquisition  Plan.  The  requirements  for  this  document  are 
enumerated  in  section  7.105  of  the  FAR.  This  plan  is  broken  down  into 
two  major  areas,  the  acquisition  background  and  objectives  portion,  and 
the  plan  of  action.  (31:97) 

The  background  and  objectives  portion  includes  descriptions  of  the 
Statement  of  Need;  applicable  conditions  for  the  contract ;  costs, 
including  life  cycle  cost,  design- to  cost,  and  the  application  of  should 
cost;  product  capability  and  performance  requirements;  delivery  or 
performance  period  requirements;  trade-off  analyses;  and  projected 
risks.  (31:97) 

The  plan  of  action  describes  potential  sources;  the  use  of 
competition;  source  selection  procedures;  contracting  considerations; 
budgeting  and  funding;  the  product  itself;  priorities,  allocations  and 
allotments;  contractor  versus  government  performance  requirements; 
management  information  requirements;  make  or  buy  considerations;  test 
and  evaluation;  logistical  considerations;  required  government  furnished 
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property  and  information;  environmental  considerations;  security 
considerations;  significant  program  milestones  and  the  major 
participants  in  the  program.  (31:97) 

Procurement  Plan.  The  contracting  officer  prepares 
the  procurement  plan,  which  simply  documents  the  procurement  portion  of 
the  acquisition  strategy.  This  plan  includes  information  concerning  the 
review  of  the  procurement  request,  the  availability  of  sources,  the 
review  and  approval  of  the  proposal  evaluation  and  source  selection 
criteria,  the  source  selection  plan,  the  determination  of  competitive 
procedures,  the  selection  of  the  type  of  contract,  the  assessment  of 
market  conditions,  the  determination  for  set-asides,  the  subcontracting 
requirements,  and  the  potential  for  involvement  of  small  and 
disadvantaged  businesses  (8A).  It  may  also  include  the  requirements  for 
acquisition  from  mandatory  sources,  the  procurement  history  of  the  item 
being  purchased,  an  identification  of  long  lead  items,  and  the 
availability  of  government  furnished  property  (GFP) .  In  addition,  it 
contains  first  article  acceptance  requirements,  financial  alternatives, 
identification  of  special  contract  provisions  and  deviations  from  the 
FAR,  assignment  of  contract  administration  functions,  and  schedule 
completion  times  for  each  task  performed  under  the  contract.  (31:20- 
22) 

Characteristics  of  Sound  Acquisition  Strategy.  Although  the 
regulatory  guidance  specifies  that  acquisition  strategy  planning  will  be 
done,  there  is  no  official  guidance  on  how  it  will  be  done.  Several 
unofficial  manuals  have  been  published,  however,  which  attempt  to 
describe  the  formulation  and  salient  features  of  the  acquisition 
strategy.  Hie  National  Contract  Management  Association's  Acquisition 
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Planning  for  Major  Systems  states  that,  in  addition  to  identifying 
program  options,  the  strategy  must  also  address  the  management  processes 
that  control  the  acquisition  cycle  (26:58).  Part  of  the  importance  of 
the  acquisition  strategy  lies  in  the  fact  that  parts  of  it  must  be 
"translated  into  definitive  requirements  which  can  be  relayed  to 
industry  through  contractual  agreements"  (26:58). 

The  development  of  an  appropriate  acquisition  strategy  is 
considered  fundamentally  important  because  it  ultimately  leads  to  the 
program  manager’s  attainment  of  program  objectives.  The  strategy 
provides  an  organized  and  consistent  approach  to  meeting  program 
objectives  within  known  constraints.  A  sound  strategy  minimizes 
technical,  schedule,  and  cost  risks  in  order  to  achieve  program 
stability.  The  acquisition  strategy  develops  technological  options, 
explores  design  concepts,  and  plans  and  conducts  acquisition  activities 
to  attain  balanced  cost  effectiveness.  Because  of  its  importance  to  the 
overall  program,  it  must  be  developed  as  early  in  the  program  life  cycle 
as  possible.  (27:3-1) 

As  a  minimum,  the  acquisition  strategy  should  address  three  major 
concern  areas  while  meeting  five  essential  criteria.  Hie  concern  areas 
are  strategic ,  which  includes  national  objectives,  the  nature  of  the 
threat,  the  need  for  the  program,  the  technology  base,  the  program’s 
objectives,  constraints  and  priorities,  the  industrial  and  political 
market  factors,  and  critical  program  issues;  technical ,  which  includes 
design,  test  and  evaluation,  production,  and  deployment;  and  resource, 
which  includes  the  personnel  and  organization  required  to  manage  the 
program,  the  schedule,  the  business  and  financial  factors,  the 
management  information  system,  and  facilities.  (27:3-5-9) 
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The  five  criteria  that  the  strategy  must  meet  in  order  to  be 
effective  are  realism,  stability,  resource  balance,  flexibility,  and 
controlled  risk.  Realism  results  in  program  objectives  which  can  be 
attained.  Without  realism,  a  program  may  find  itself  in  constant 
turmoil  and  crisis  and  possibly  in  failure.  Realism  is  achieved  by 
doing  as  much  homework  as  possible  on  the  characteristics  and 
environment  and  planning  accordingly. 

Stability  does  not  allow  internal  and  external  influences  to 
seriously  disrupt  the  program.  Instability  can  lead  to  reduced 
confidence  in  program  estimates  and  assumptions,  increased  government 
and  contractor  risk,  and  reduced  morale  and  motivation.  Stability  can 
be  achieved  through  commitment,  direction,  and  advocacy.  (27:3-9-14) 
Resource  balance  results  in  a  program  in  which  all  of  the  risks 
are  approximately  equal.  A  lack  of  balance  leads  to  the  overemphasis  of 
one  objective  at  the  expense  of  another,  and  could  ultimately  result  in 
not  meeting  that  other  objective.  Balance  can  be  achieved  through  the 
allocation  of  resources  in  a  way  which  attains  the  required  level  of 
capability  without  changes  in  resource  requirements.  (27:3-15-16) 
Flexibility  is  the  program’s  ability  to  absorb  changes  and 
failures  without  adversely  effecting  resource  requirements . 

Inflexibility  can  result  in  an  unbalanced  and  unstable  program, 
unrealistic  approaches  and  spiraling  management  problems.  Flexibility 
can  be  achieved  by  identifying  early  on  those  areas  where  changes  and 
failures  are  highly  probable  and  developing  contingency  plans.  (27:3- 
16-17) 

Risk  is  a  measure  of  the  probability  and  consequence  of  not 
achieving  a  defined  program  goal.  Controlling  risk  can  be  accomplished 
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by  structuring  the  acquisition  strategy  to  identify  hazards  and  allow 
for  the  development  of  safeguards  to  overcome  than.  (27:20) 

A  program  manager  who  can  develop  an  acquisition  strategy  early  on 
which  addresses  the  major  concern  areas  while  meeting  the  essential 
criteria  of  good  strategy  will  be  well  along  the  way  to  achieving 
program  success.  The  sheer  scope  of  the  strategy,  however,  indicates 
that  it  cannot  be  developed  in  a  vacuum.  The  program  manager  must  fully 
utilize  the  functional  experts  available  to  him  or  her.  In  addition 
corporate  expertise  and  experience  can  be  brought  to  bear  to  assist  the 
program  manager  by  allowing  for  the  input  of  senior  level  acquisition 
personnel.  The  Air  Force  has  adopted  the  Acquisition  Strategy  Panel 
approach  to  do  this. 

Implementation  Issues.  The  information  presented  in  previous 
sections  of  this  chapter  point  out  the  complexity  of  the  acquisition 
strategy.  This  complexity  has  led  to  several  problems  in 
implementation .  According  to  Bissett,  in  his  Naval  Post  Graduate  School 
thesis,  a  broad  acquisition  strategy  for  most  major  programs  is 
developed  during  preparation  of  the  Mission  Need  Statement  (MNS).  The 
MNS  is  developed  at  the  headquarters  level  of  the  military  branch  and  is 
done  prior  to  the  assignment  of  a  dedicated  program  manager.  Major 
Bissett  argues  that  the  program  manager  (FM)  does  not  actually  develop 
the  program’s  acquisition  strategy,  but  rather  is  constrained  by  a  broad 
strategy  which  he  may  not  like  and  whose  development  he  did  not 
participate  in.  At  this  point  the  program  manager  may  only  be  able  to 
expand  on  the  basic  strategy  which  has  already  been  defined.  Most  of 
the  acquisition  personnel  that  Bissett  interviewed  "...  felt  that  the 
overall  framework  for  the  program  had  already  been  developed  before  the 
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FM  had  the  opportunity  to  develop  his  acquisition  strategy.”  Thus,  they 
".  .  .  viewed  the  FM's  role  as  one  of  implementing  the  overall  strategy 
developed  by  higher  levels  ..."  (3:45-47) 

In  1985,  Swanson  studied  the  execution  of  acquisition  strategies. 
His  study  found  execution  problems  in  the  areas  of  schedules,  and 
technologies.  Schedule  problems  included  a  lack  of  integrated  planning, 
having  to  adhere  to  ambitious  schedules  that  were  established  outside  of 
the  program  office  and  changing  lead  times  which  led  to  delivery  delays. 
The  problems  faced  by  program  managers  the  most  in  the  technological 
area  were  baseline  changes  to  requirements,  technology  transfer  issues 
and  lower  then  predicted  performance.  Other  problems  cited  included 
cost  growth  and  the  turnover  of  program  office  personnel.  The 
implication  here  is  that  the  program  manager  should  consider  these 
problems  when  developing  the  program  strategy.  (30:59-61) 

The  study  also  pointed  to  several  strategies  associated  with 
successful  programs.  In  the  area  of  cost,  the  use  of  multi-year 
funding,  program  unique  inflation  rates  and  cost  realism  as  a  source 
selection  criteria  improved  strategy  execution.  The  use  of  the  critical 
path  method,  concurrent  design  refinement  and  testing,  and  a  consensus 
regarding  initial  operating  capability  definition  and  date  all  improved 
strategy  execution  in  the  area  of  schedule  performance .  To  improve  the 
technical  aspect  of  strategy  execution,  the  study  suggests  the  use  of 
tailored  and  thin  requests  for  proposal,  realistic  technical  risk 
evaluations  to  justify  the  type  of  contract  used,  and  actual  operational 
experience  to  change  the  technical  baseline.  Organizationally, 
coordination  among  the  major  program  participants  to  ensure  realism  and 
overcome  organizational  and  personal  bias ,  and  frequent  informal  reviews 
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of  the  acquisition  strategy  to  foster  commitment  and  advocacy  among  the 
program  participants  were  suggested  as  ways  to  improve  strategy 
execution.  (30:61-64) 

This  section  has  highlighted  the  importance  and  characteristics  of 
sound  acquisition  strategy.  Air  Force  Systems  Command  has  formalized 
the  process  by  implementing  the  Acquisition  Strategy  Panel  (ASP)  which 
is  discussed  in  the  next  section. 

The  Acquisition  Strategy  Panel 

In  AFSC  Regulation  550-21,  Commander’s  Policies:  Acquisition 

Strategy,  published  in  1987,  General  Bernard  Randolph,  former  Air  Force 

Systems  Comnand  Commander,  stated  that  "Effective,  timely  acquisition 

strategy  is  ...  our  primary  weapon  against  future  program  cost, 

schedule,  and  management  problems"  (21:2).  Those  words  emphasized  the 

importance  of  strategic  planning  within  the  command.  The  role  of  the 

command’s  headquarters  staff  was  made  clear  when  he  stated  "I  consider 

the  initial  formulation  and  maintenance  of  effective  strategy  to  be  key 

responsibilities  of  my  commanders  and  my  staff"  (21:2).  To  allow  for 

headquarters  involvement  in  program  strategy  development,  the  regulation 

called  for  the  establishment  of  "Acquisition  Strategy  Panels"  (ASP)  as 

the  key  tool  for  defining  acquisition  strategy  in  AFSC  (21:1).  This 

involvement  by  the  command’s  headquarters  staff  is  expected  to  continue 

under  the  leadership  of  the  current  AFSC  Comnander ,  General  Ronald 

Yates,  who  indicated  his  support  by  stating: 

The  headquarters  .  .  .  has  the  very  real  responsibility  to  .  .  . 
develop  the  early  critical  program  goals  in  terms  of  requirements 
and  program  acquisition  strategies.  How  these  factors  are  laid 
out  early  on  ultimately  determines  the  success  of  a  program. 
(22:14) 
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Hie  policy  statement  delineates  the  essential  criteria  for  all 
ASPs.  Among  these  criteria  are  planning  for  acquisition  strategy  early 
in  the  program  life  cycle,  focusing  the  best  talent  available  on  the 
development  of  the  acquisition  strategy,  inmediately  addressing  any 
issues  surfaced  during  the  creation  of  the  strategy  to  avoid  later 
problems,  ensuring  that  the  system  fully  meets  the  user’s  needs,  paying 
special  attention  to  the  timing  of  program  phases,  and  using  the  end 
product,  the  Acquisition  Plan,  as  the  roadmap  for  all  future  program 
activity.  In  the  policy  statement,  it  is  made  clear  that  this  is  a 
responsibility  of  the  headquarters  staff,  rather  than  the  product 
divisions  or  the  individual  program  managers.  (21:1-2) 

Hie  ASP  grew  out  of  the  Business  Strategy  Panel.  (BSP)  which  was 
used  by  the  Air  Force  until  1987.  In  1986,  the  BSP  was  changed  from  a 
strictly  contractual  review  to  a  two-session  briefing  that  looked  not 
only  at  the  contractual  issues  of  the  program,  but  also  at  the  program's 
product  assurance  and  acquisition  logistics  issues.  Hie  ASP,  then,  is 
the  direct  descendant  of  the  two-session  BSP.  Hie  major  difference  is 
that,  while  the  BSP  dealt  primarily  with  the  contractual  and  business 
aspects  of  the  program,  the  ASP  takes  a  broader  perspective  by  looking 
at  the  total  acquisition  approach,  to  include  engineering,  support  and 
test  issues.  There  are  other  differences  between  the  BSP  and  the  ASP. 
Hie  chairpersons  of  the  BSP  were  appointed  by  regulation  while  the  ASP 
chairperson  is  appointed  by  the  AFSC  Commander  and,  thus,  works  directly 
for  the  conmander.  Hie  BSP  relied  upon  ad-hoc  committee  membership 
whereas  the  ASP’s  membership  is  made  up  of  a  standing  panel.  And 
finally,  ASP  guidance  makes  provisions  for  all  programs  while  the  BSP 
focused  solely  on  major  programs.  (12) 
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What  is  an  ASP?  Specific  ASP  guidance  iB  set  forth  in  AFSC 
Regulation  800-53,  Acquisition  Strategy  Panels.  The  ASP  is  a  corporate 
review  of  a  program  manager's  strategy  by  a  standing  panel  of  top  AFSC 
experts.  It  provides  him  or  her  with  advice  on  the  development  of  an 
executable  acquisition  strategy  which  meets  the  user's  needs.  The  end 
product  of  the  ASP  is  a  workable  Acquisition  Plan.  (19:1) 

The  Purpose  of  the  ASP.  The  objective  of  the  ASP  is  to 
bring  "the  experience  and  viewpoints  of  AFSC's  senior  acquisition 
managers  {to  bear)  in  a  systematic  process  during  the  formulation  and 
selection  of  program  acquisition  strategies"  (19:1).  According  to 
briefing  charts  used  by  HQ  AFSC  personnel  to  explain  the  ASP  process, 
there  are  several  other  objectives.  These  include  the  efficient  use  of 
the  command's  resources,  the  integration  of  functional  disciplines  in 
the  program  planning  and  execution  process,  and  the  application  of 
lessons  learned  across  the  command  (12). 

The  ASP  was  created  in  1987  amid  growing  concern  that  program 
managers  were  not  getting  the  help  they  needed  in  developing  complex 
acquisition  strategies.  At  a  workshop  on  acquisition  strategy  held  at 
the  Defense  Systems  Management  College  (DSMC)  in  1984,  it  was  felt  that 
".  .  .  acquisition  managers  {had}  a  valid  need  for  guidance  to  develop 
sound  acquisition  strategies  for  new  programs  and  to  adjust  the  .  .  . 
strategies  for  existing  programs  as  they  {proceeded}  through  the  life 
cycle"  (1:13).  At  about  the  same  time,  Bissett  noted  that  "There 
{appeared}  to  be  a  lack  of  clear  direction  in  the  actual  development  of 
program  acquisition  strategies"  (3:40).  The  ASP  was  an  attempt  by  the 
Air  Force  to  fill  a  part  of  this  void. 
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In  addition  to  needing  proper  guidance,  the  program  manager  also 
had  .  .a  need  to  incorporate  the  policies,  procedures  and  strategy 
of  higher  levels  when  considering  the  options  available  .  .  ."in 
selecting  a  strategy  (3:27).  It  was  often  the  case  that  the  program 
manager's  strategy  did  not  match  the  desires  of  higher  level  managers . 
Indeed,  a  key  to  the  process  of  selecting  the  appropriate  strategy  for  a 
program  was  "...  the  early  identification  of  higher  level  strategies, 
objectives,  priorities  and  policies"  (3:39).  By  involving  the  higher 
level  headquarters  staff  in  the  strategy  selection  process,  it  was 
proposed  that  many  of  these  concerns  would  be  eliminated. 

In  an  analysis  of  strategic  planning  among  businesses,  Kerzner 
found  that  most  of  the  tasks  involved  lent  themselves  to  being  done  by 
the  program  manager,  but  there  were  several  which  required  the  expertise 
and  insight  of  the  executive  level.  These  tasks  included  setting 
overall  objectives  by  establishing  where  the  firm  was  at  the  time  and 
where  it  wanted  to  be  in  the  future,  and  selecting  a  strategy  portfolio 
which  included  an  analysis  of  the  best  course  for  the  firm  to  take  as 
well  as  the  potential  risks  and  benefits  of  the  strategy.  Tasks  left  to 
the  program  manager  and  his  staff  included  an  environmental  analysis  of 
the  firm's  business  climate,  listing  alternative  strategies  which 
followed  the  establishment  of  objectives  by  the  executive  level,  listing 
threats  and  opportunities,  preparing  forecasts  followed  by  the  selection 
of  the  strategic  portfolio  by  the  executive  level,  preparing  action 
plans,  and  monitoring  and  controlling  the  program  once  selected. 

(23:471) 

While  these  tasks  were  broken  out  between  the  program  manager  and 
the  executive  level,  Kerzner  maintains  that  the  program  manager  involved 
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in  strategic  planning  should  be  able  to  report  directly  to  the  executive 
level  and  should  have  the  authority  required  to  obtain  the  resources 
required.  The  nature  of  strategic  planning  itself  requires  a  close 
working  relationship  between  the  executives  and  the  program  manager. 
Indeed,  "Top-management  involvement  may  very  well  be  the  single  most 
important  variable  in  strategic  planning."  (23:470-475)  All  of  the 
points  made  by  Kerzner  fall  in  line  with  what  the  ASP  attempts  to  do  for 
the  Air  Force. 

Another  unstated  purpose  of  the  ASP  may  be  its  use  as  a  vehicle  to 
build  a  consensus  of  support  for  the  program.  According  to  Swanson, 

"the  changing  program  direction  imposed  on  the  program  manager  from 
outside  agencies  {was}  one  of  the  most  difficult  challenges  for  most 
program  managers"  (30:62).  Bissett,  in  characterizing  the  problem, 
stated  that 

.  .  .  the  FM  must  satisfy  many  organizations  and  individuals  who 
may  not  be  able  to  appreciate  the  overall  strategy  of  the  program. 
As  a  result,  there  are  many  individuals  who  can  say  "no"  to  a 
particular  portion  of  the  acquisition  strategy.  There  are  few 
individuals,  however,  who  can  say  "yes"  to  the  overall  strategy. 
(3:44) 

Bringing  many  of  the  headquarters  personnel  who  had  the  authority  to  say 
"no"  to  a  portion  of  the  strategy  together  in  one  room  to  review  the 
strategy  at  the  same  time  through  an  ASP  could  be  effective  at  reducing 
dissent  for  the  final  approved  strategy.  The  approval  of  an  acquisition 
strategy  by  the  appropriate  decision  authority  could  further  reduce  the 
problem  of  dissent  by  serving  ".  .  .  as  a  formal  agreement  between  the 
FM  and  the  decision  authority  relative  to  how  the  acquisition  {would}  be 
accomplished"  (3:62).  Additionally,  a  well  documented  strategy  could  be 
used  later  in  the  program  to  ".  .  .  review  the  effect  of 
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counterproductive  direction”  and  establish  a  firm  ground  for  resisting 
their  implementation  (30:62). 

One  of  the  keys  to  success  is  for  the  FM  to  have  an  appreciation 
of  the  impact  that  the  political  process  could  have  on  the  program 
(3:47).  Accordingly,  another  unstated  purpose  of  the  ASP  may  lie  in  the 
fact  that  the  headquarters  staff,  based  in  Washington  D.C.,  is  in  a 
unique  position  to  provide  valuable  political  insight  that  "light 
otherwise  be  unavailable  to  the  program  manager.  This  insight,  when 
factored  into  the  early  strategy  of  the  program,  could  result  in  fewer 
political  problems  later  on. 

The  ASP  can  be  seen,  then,  as  a  vehicle  to  fill  a  perceived  gap  in 
strategic  direction  for  the  program  manager  and  to  incorporate  the 
policies  and  strategies  of  higher  level  decision  makers.  It  could  also 
be  used  to  build  a  consensus  of  support  for  the  program  early  on  in 
order  to  reduce  dissent  later  in  the  program  life  cycle  and  to  provide 
the  program  manager  with  insight  into  the  politics  which  might  affect 
the  program. 

Characteristics  of  the  ASP.  ASPs  will  be  conducted  by  AFSC  for 
programs  which  are  forecast  to  require  more  than  $200  million  in 
research  and  development  funding  or  $1  billion  in  production  funding, 
$750  million  in  modification,  maintenance,  or  service  cost,  or  high 
visibility  or  special  interest  acquisitions  as  designated  by  AFSC.  An 
ASP  is  conducted  every  time  one  of  these  programs  approaches  a  new 
milestone  or  phase  in  the  acquisition  life-cycle.  Additionally,  an  ASP 
may  be  convened  at  the  request  of  the  program  manager  if  the  program 
undergoes  a  significant  change  in  funding  or  direction  which  would  cause 
a  fundamental  change  in  its  acquisition  strategy.  ASPs  are  also 
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conducted  for  smaller  programs  with  the  panel  members  being  drawn  from 
the  product  division  or  program  office  staffs.  Figure  3  illustrates  the 
thresholds  and  levels  at  which  an  ASP  may  be  conducted.  (19:1) 
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Figure  3.  Thresholds  for  ASP  Levels 


The  ASP  is  conducted  prior  to  the  publication  of  the  formal 
acquisition  plan,  but  not  before  the  program  has  been  defined  in  a 
statement  of  requirements  or  the  issuance  of  a  Program  Managcr'ent 
Directive  (FMD).  The  ASP  process  is  conducted  in  the  form  of  a  briefing 
by  the  program  manager,  or  his  functional  experts,  to  the  standing  panel 
members.  The  briefing  includes  discussions  of  requirements, 
engineering,  test,  support,  budget,  and  business  issues.  During  the 
briefings  the  panel  members  will  provide  comments  on  the  approaches 


being  taken  by  the  program  office  and  may  suggest  alternatives.  With 
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the  publication  of  formal  minutes,  the  program  manager  is  required  to 
follow  up  with  a  report  to  the  ASP  chairperson  on  how  the 
recommendations  will  be  resolved.  (19:2) 

Implementation  of  the  ASP.  Hie  ASP  is  one  step  in  the  total 
acquisition  process  of  a  program.  While  the  format  of  the  ASP  for  major 
programs  is  relatively  constant,  the  implementation  varies  slightly  from 
product  division  to  product  division.  At  Aeronautical  Systems  Division 
(ASD) ,  AFSC’s  largest  product  division,  the  ASP  is  one  of  several 
reviews  that  lead  to  the  release  of  a  request  for  proposal  (RFP)  for  a 
particular  phase  of  a  program.  The  ASD  process  will  be  illustrated  as 
an  example. 

The  Acquisition  Planning  and  Review  Process.  Initially,  an 
Acquisition  Review  Team  (ART)  is  convened  at  the  program  manager’s 
option.  This  ART  Phase  I  is  used  to  provide  product  division  level 
advisory  assistance  to  the  PW  for  putting  the  acquisition  strategy 
together.  Upon  receipt  of  the  IWD,  the  ASP  is  convened  to  provide  a 
corporate  review  of  the  overall  program  strategy.  Its  primary  output  is 
the  acquisition  plan.  This  is  followed  by  the  ART  Phase  II  which 
provides  a  complete  and  independent  technical  and  business  review  of  the 
solicitation  prior  to  release  to  industry.  Next,  a  Source  Selection 
Management  Group  (SSMG)  is  convened  to  review  the  P^l’s  source  selection 
plans,  standards  and  policies.  At  the  end  of  this  process,  the  RFP  is 
released.  Figure  4  illustrates  this  process.  (18:5) 

The  ASP  Process.  The  ASP  is  mandatory  for  Air  Force  major 
programs.  The  ASP  briefing  is  scheduled  through  the  product  division 
and  major  command  ASP  secretariats.  Oily  appointed  panel  members, 
personnel  invited  by  the  ASP  chairperson,  and  program  office  personnel 
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attend  the  ASP.  Panel  members  are  appointed  by  the  AFSC  Commander.  The 
acquisition  strategy  briefing  is  typically  presented  by  the  program 
manager  or  his  designated  representative  and  covers  six  major  functional 
areas.  It  includes  a  detailed  discussion  of  the  language  the  program 
office  expects  to  use  in  the  solicitation  and  statement  of  work  which 
will  make  up  the  RFP.  Panel  members  are  expected  to  offer  advice  and 
quidance  on  the  strategy  as  appropriate.  Prior  to  the  end  of  the 
briefing,  recommendations  are  reviewed  for  possible  inclusion  in  the  ASP 
minutes.  Upon  receipt  of  the  minutes,  the  program  manager  has  20 
working  days  to  resolve  how  the  recomuendat ions  will  be  implemented  and 
respond  back  to  the  ASP  chairperson  in  writing.  (19:1-2) 
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The  ASP  is  made  up  of  a  standing  body  of  experienced  acquisition 
personnel  from  both  the  major  command  and  product  division  levels. 

Panel  membership  at  the  AFSC  level  includes  the  chairperson  and 
representatives  from  the  program  management,  acquisition  logistics, 
competition  advocate,  comptroller,  contracting,  legal,  product 
assurance,  engineering,  technology,  test  resources,  and  computer 
engineering  functions.  The  currently  appointed  chairperson  of  the  ASP 
is  the  AFSC  Principal  Assistant  to  the  Deputy  Chief  of  Staff  for 
Contracting.  Membership  at  the  product  division  level  parallels  that  of 
AFSC.  The  ASP  is  co -chaired  by  the  AFSC  and  product  division 
chairpersons.  In  addition,  representatives  from  the  using  command  and 
the  Office  of  the  Secretary  of  the  Air  Force  typically  attend  the  ASP 
briefings.  Other  offices  which  have  special  interests  in  specific 
programs  are  invited  on  a  case  by  case  basis.  Figure  5  illustrates  the 
composition  of  the  panel.  (19:2) 

Items  Covered  in  the  ASP.  The  acquisition  strategy  briefed 
by  the  program  manager  is  broken  into  seven  major  sections.  Each  of 
these  sections  is  described  in  detail  in  the  ASP  Presentation  Guide 
published  by  AFSC.  The  program  manager  is  expected  to  roughly  follow 
the  guide  while  tailoring  the  briefing  to  his  particular  program.  The 
first  section  is  a  description  of  the  program  and  its  requirements  and 
serves  to  insure  that  the  program  office  and  user  requirements  coincide. 
The  second  section  describes  the  engineering  and  technical  aspects  of 
the  program,  specifically  the  technical  risks  associated  with  the 
program  and  the  strategy  for  dealing  with  those  risks  from  a  program 
life  cycle  standpoint.  The  third  section  addresses  the  program's  test 
resource  and  facilities  requirements  as  well  as  the  success  and  failure 
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ACQUISITION  STRATEGY  PANEL  COMPOSITION 
AT  HQ  *CSC 
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Figure  5.  Headquarters  AFSC  Acquisition  Strategy  Panel  Composition 


criteria  for  the  test  program.  The  fourth  section  deals  with  support 
issues  which  the  program  will  face  upon  the  fielding  of  the  weapon 
system  and  is  geared  toward  logistical  concerns.  The  fifth  section 
discusses  budget  requirements,  focusing  on  funding  required  and  timing 
for  those  funds.  The  sixth  section  deals  with  the  business  aspects  of 
the  program  by  focusing  on  the  source  selection  and  solicitation  process 
as  well  as  business  base  considerations.  The  last  section  is  a  summary 
of  critical  issues  present  in  the  program’s  strategy  and  recommendations 
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for  action  required  outside  of  the  program  office.  Appendix  A  shows 
each  of  the  major  sections  and  the  specific  topics  to  be  covered  under 
each.  (13:3-17) 

The  ASP  in  Action.  The  issue  of  whether  or  not  the  ASP  adds 
value  for  the  program  manager  appears  to  be  a  concern  of  AFSC's.  An 
initiative  was  undertaken  to  reduce  duplication  of  effort  in  the  ASP 
process.  The  command  published  a  guide  for  writing  effective  minutes 
and  produced  a  videotape  discussing  the  ASP  process.  The  command  went 
on  to  identify  what  it  considered  to  be  keys  to  ASP  success,  as  well  as 
challenges  faced  by  the  command  to  insure  the  ASP’s  continued  value. 
Finally,  the  command  sought  the  opinions  of  program  managers  to  improve 
ASP  effectiveness.  The  next  several  paragraphs  discuss  each  of  these 
initiatives  in  more  detail. 

One  of  the  initial  problems  associated  with  the  ASP  was  that 
program  managers  were  being  forced  to  present  their  strategies  at  two 
levels,  the  product  division  headquarters  and  AFSC  headquarters.  The 
result  was  duplication  of  effort  and  conflicting  guidance  from  the 
separate  panels.  In  August  1989,  based  upon  input  received  from  the  his 
commanders,  the  AFSC  commander  sent  a  message  to  all  of  the  product 
divisions  implementing  "Joint  ASPs"  as  a  way  of  eliminating  the  problem . 
The  joint  ASPs  would  be  held  at  the  product  division  with  both  the  AFSC 
and  product  division  panels  in  attendance  and  would  be  co -chaired  by  the 
AFSC  and  the  product  division  ASP  chairpersons.  This  eliminated  the 
need  for  the  program  manager  and  his  staff  to  travel  to  AFSC 
headquarters ,  thus  reducing  the  amount  of  travel  the  program  office  was 
required  to  make,  as  well  as  eliminating  the  problem  of  conflicting 
guidance.  In  addition,  General  Randolph  emphasized  that  the  program 
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manager  was  to  concentrate  on  developing  his  acquisition  strategy,  not 
his  briefing,  by  saying  "We  do  not  conduct  ASPs  to  produce  polished 
briefings  but  to  assist  program  directors  in  forging  a  strategy  to 
produce  an  executable  program."  (2:2) 

Another  problem  was  confusion  in  the  ASP  preparation  process.  To 
remedy  the  situation,  AFSC  published  a  handbook  for  its  panel  members, 
distributed  a  preparation  guide  to  be  used  by  the  affected  program 
offices,  began  publishing  a  letter  semi-annually  which  described  lessons 
learned  from  other  ASPs,  instituted  a  training  program  for  new  panel 
members,  and  conducted  visits  to  the  product  divisions  to  discuss  ASPs 
in  general.  For  each  ASP,  a  pre-panel  overview  meeting  was  held  among 
the  panel  members  to  discuss  the  issues  likely  to  surface  during  the  ASP 
itself,  an  executive  summary  was  presented  to  the  AFSC  Commander  within 
48  hours  of  the  ASP,  minutes  with  recommendations  were  published  and 
responses  to  those  recommendations  were  solicited  from  the  program 
directors.  (12) 

Since  well  written  minutes  of  the  briefing  were  considered  vital 
to  the  usefulness  of  any  ASP,  AFSC  published  a  guide  to  assist  personnel 
responsible  for  writing  those  minutes.  The  minutes  served  as  a  track 
record  of  the  panel’s  discussion  and  recommendations.  Those 
recommendations  had  to  either  be  included  in  the  program’s  acquisition 
plan  or  responded  to,  in  writing,  by  the  program  manager.  The  guide 
included  the  basic  format  for  writing  minutes,  hints  on  what  to  write 
and  how  to  write  than  well,  and  examples  of  the  various  documentary 
products  of  the  ASP.  The  guide  called  upon  the  writer  of  the  minutes  to 
review  the  acquisition  plan  to  insure  that  the  strategy  in  the  plan  was 
consistent  with  the  strategy  which  was  agreed  to  during  the  ASP.  (14) 
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To  further  assist  those  personnel  who  were  new  to  the  process, 

AFSC  commissioned  a  videotape  which  introduced  the  ASP  concept  and 
discussed  its  features.  The  tape  started  with  an  introduction  by 
General  Randolph.  In  the  introduction,  he  stated  that  the  ASP  was  a  way 
to  bring  the  right  people  together  at  the  right  time  to  offer  advice, 
lessons  learned  and  insight  to  the  program  manager.  He  stated  that  the 
ASP  was  not  to  be  viewed  as  a  bureaucratic  hurdle,  but  as  a  "dynamic 
creative  effort  to  solve  problems."  The  tape  highlighted  the  way  the 
ASP  worked  and  illustrated  the  results  of  some  past  ASPs  through  the 
comments  of  six  program  managers  who  had  recently  been  through  the 
process .  ( 20 ) 

According  to  AFSC,  there  were  several  keys  to  a  successful  ASP. 
These  keys  included  having  good  people  as  members  of  the  panel.  Panel 
membership  should  include  functional  experts  as  well  as  highly 
experienced  acquisition  personnel.  Also  key  was  the  adequate 
preparation  by  SPO  personnel  and  panel  members  prior  to  the  ASP. 
Discussion  during  the  ASP  itself  was  to  flow  both  ways  and  should 
include  a  discussion  of  the  program’s  tough  issues  to  be  of  any  real 
value.  Feedback  has  also  been  highlighted  by  AFSC  as  being  vital  to  the 
process.  (12) 

The  command  has  identified  several  other  problem  areas  which 
required  special  attention  to  ensure  that  the  ASP  remained  a  valuable 
tool.  These  included  the  potential  for  poor  quality  panels  as  a  result 
of  a  lack  of  panel  membership  stability  and  a  lack  of  representation  of 
all  functional  areas.  Scheduling  was  another  problem  which  the  command 
felt  could  be  addressed  by  providing  program  offices  with  sufficient 
lead  time  to  allow  for  adequate  preparation  and  by  coordinating 
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schedules  to  allow  for  participation  by  personnel  from  the  office  of  the 
Secretary  of  the  Air  Force.  Finally,  the  command  identified  follow-on 
tracking  as  necessary  to  ensure  that  panel  recomnendations  were 
addressed  and  that  the  long-term  success  or  failure  of  the  strategies 
were  evaluated.  (12) 

In  1988  and  1989,  AFSC  sought  the  input  of  program  managers  who 
had  been  through  the  ASP  process  regarding  its  value  and  suggestions  for 
improvements.  This  input  was  solicited  in  meetings  between  command  and 
field  personnel  at  the  product  divisions.  Questions  such  as  "has  the 
ASP  process  helped  the  program  manager  develop  a  more  effective 
acquisition  strategy?"  and  "are  program  directors  receiving  proper 
level (s)  of  expertise  at  field  level  ASPs?"  were  asked.  While  the 
author  found  no  direct  evidence  of  the  results  of  these  discussions,  a 
chart  used  by  command  personnel  to  describe  the  process  to  higher  level 
personnel  stated  that  program  directors  had  mixed  reactions.  According 
to  the  chart,  "seme  PD’s  (were)  very  positive"  while  "same  {saw  the) 
process  as  an  obstacle."  One  of  the  recommendations  resulted  in  the 
establishment  of  the  joint  ASP  discussed  earlier.  (12) 

Part  of  this  push  to  enhance  the  image  of  the  ASP  is  evident  in  a 
briefing  chart  addressed  to  program  managers  entitled  "How  should  you 
view  ASPs?  (Dispelling  Misconceptions) .0"  The  chart  compared  what  ASPs 
were  to  what  they  were  not.  According  to  the  chart,  ASPs  were  a  dynamic 
process,  a  forum  for  new  ideas,  a  resource  to  be  used  by  the  program 
directors,  and  a  consulting  service.  The  chart  stated  that  ASPs  were 
not  bureaucratic  hurdles,  pass  or  fail  exercises,  square  fillers  or 
miniature  inspector  general  inspections.  The  chart  went  on  to  state 
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that  teamwork  between  the  program  office  and  the  panel  members  was 
essential.  (12) 

The  previous  paragraphs  indicate  that  AFSC  was  concerned  about  the 
image  of  its  ASP  process  among  the  working  program  managers  in  the 
field.  The  public  relations  campaign  embarked  upon  by  AFSC  which  has 
resulted  in  the  publication  of  guides,  production  of  a  videotape,  and 
charts  presented  by  the  command  shows  that  support  for  the  ASP  has  been 
questionable  in  the  past.  Many  improvements  in  the  process  have  been 
made,  as  evidenced  by  the  implementation  of  the  joint  ASP.  Indeed,  Mrs 
Darleen  Druyun,  the  AFSC  Principal  Assistant  to  the  Deputy  Chief  of 
Staff  for  Contracting,  received  an  Executive  Achievement  Award  in  1990 
for  "her  initiative  and  direction  in  establishing  the  Acquisition 
Strategy  Panel  concept"  (25:20).  The  question  of  the  value  of  the  ASP 
to  the  program  manager,  however,  remains  unanswered.  The  concern 
demonstrated  by  AFSC  regarding  program  managers'  perceptions  of  the  ASP 
argues  for  a  closer  examination  of  the  process. 

An  interesting  recent  development  in  the  strategy  review  process 
was  contained  in  a  memorandum  distributed  by  the  Assistant  Secretary  of 
the  Air  Force  (Acquisition)  on  4  April  1990.  In  this  memorandum,  the 
assistant  secretary  called  for  the  "institutionalizing"  of  a 
Requirements  and  Acquisition  Program  Review  process  for  all  Air  Force 
major  programs.  This  process  grew  out  of  the  success  of  the  "Svnanit" 
meetings  held  to  review  the  B-1B,  C-17,  Advanced  Tactical  Fighter,  and 
B-2  aircraft  programs.  The  reviews,  to  be  held  between  milestones  I, 

II,  and  III,  would  have  a  threefold  focus.  According  to  the  memo,  this 
focus  would  include  the  examination  of  operating  coranand  requirements 
relative  to  the  projected  threat  and  operational  concept,  a  review  of 
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program  progress  toward  accomplishing  those  requirements,  and  the 
establishment  of  realistic,  achievable  and  affordable  performance  goads 
and  thresholds  as  a  function  of  time.  These  reviews  would  be  chaired  by 
the  Air  Force  Chief  of  Staff.  (28) 

This  new  review  process  appears  to  follow  the  original  objectives 
of  the  ASP  fairly  closely.  This  will  most  likely  result  in  further 
duplication  of  effort  within  the  Air  Force  bureaucracy,  which  is 
precisely  what  it  cannot  afford  at  this  time.  Today  the  DoD  and  Air 
Force  sire  being  faced  with  increasing  pressure  to  cut  budgets  and 
streamline  acquisition  processes.  With  those  mounting  pressures,  the 
Air  Force  can  ill  afford  to  spend  its  time  in  inefficient  processes  or 
those  which  do  not  provide  adequate  value  for  the  effort  expended.  The 
next  section  of  this  chapter  discusses  seme  of  the  pressures  currently 
facing  the  DoD  in  general  and  the  Air  Force  in  particular. 

The  Call  for  Reform 

This  section  serves  as  a  cursory  overview  of  some  of  the  more 
recent  reform  initiatives  undertaken  by  the  DoD  and  the  USAF.  Those 
readers  with  background  in  this  area  may  go  on  to  the  final  section  of 
this  chapter. 

Complaints  about  the  state  of  both  the  DoD  system  and  the  Air 
Force  system  are  not  rare.  The  incorporation  of  many  of  the  Packard 
Commission  Report’s  recommendations  and  volumes  of  legislation  imposed 
by  Congress  have  not  quieted  the  critics.  Many  insist  that  the  problems 
persist.  At  a  meeting  of  the  Air  Force  Studies  Board3  in  November  of 

3 The  board  was  established  in  1962  at  the  request  of  the  AFSC  Commander 
and  is  made  up  of  civilian  and  ex-military  personnel  with  experience  in  weapons 
acquisition  (17:30). 
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1987,  the  acquisition  system  was  characterized  as  being  bogged  down  by 
unnecessary  administrative  bureaucracy  and  congressional  micromanagement 
(17:30). 

There  have  been  other  calls  for  change.  These  include 
establishing  a  centralized  DoD  buying  agency  similar  to  those  in  use  in 
Britain  and  France.  Proponents  of  this  idea  feel  that  this  will  result 
in  better  weapons  for  less  cost  while  attracting  a  professional  corps  of 
acquisition  officials.  Critics  argue  that  the  current  system  produces 
the  best  weapons  in  the  world  at  reasonable  cost  and  should  not  be 
changed.  Additionally,  they  argue  that  a  single  agency  will  be  no  more 
immune  to  corruption  then  the  current  system  and  that  establishing  a 
separate  buying  entity  will  reduce  communications  with  the  using 
commands,  possibly  resulting  in  the  fielding  of  ineffective  weapon 
systems.  (16:10) 

The  Air  Force  has  also  received  its  share  of  criticism  when  it 
comes  to  acquisition.  Feeling  the  heat  the  most  is  Air  Force  Systems 
Command.  There  have  been  calls  to  eliminate  Systems  Command  altogether 
(32:80).  General  Randolph  countered  that  the  infrastructure  the  command 
is  responsible  for  would  be  unmanageable  by  someone  in  the  Pentagon  and 
that  AFSC’s  role  as  an  advocate  of  new  technology  through  the  management 
of  some  of  the  nation’s  foremost  laboratories  would  be  lost  (32:81). 

The  calls  for  reform  have  been  many  and  loud.  The  Department  of 
Defense  and  the  US  Air  Force  are  working  toward  a  more  effective  and 
efficient  acquisition  system.  TCiese  efforts  are  reviewed  below. 

DoD  and  Air  Force  Reform  Initiatives.  Hie  allegations  of 
inefficiencies  in  Air  Force  and  Department  of  Defense  procurement  have 
prompted  both  to  take  major  steps  toward  improving  the  process.  Many  of 
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these  steps  have  already  been  taken,  yet  more  needs  to  be  done.  This 
section  will  first  look  at  Air  Force  efforts  of  the  past  few  years  to 
correct  the  problems ,  then  examine  existing  DoD  policy  on  Acquisition 
Streamlining  and  conclude  with  an  examination  of  the  far  reaching 
recommendations  of  the  recently  completed  Defense  Management  Review. 

The  Air  Force  has  not  been  idle  in  the  area  of  acquisition  reform. 
A  streamlined  acquisition  management  structure  was  implemented  following 
the  publication  of  the  Packard  Commission  report.  This  structure 
consists  of  the  Air  Force  Acquisition  Executive  (AFAE)  who  has  overall 
responsibility  for  the  Air  Force’s  acquisition  system,  the  Program 
Executive  Officer  (PEO)  who  is  responsible  for  the  implementation  of 
program  direction,  the  program  manager  who  is  responsible  for  the  day- 
to-day  management  of  the  program,  and  the  commanders  of  AFSC  and  AFLC 
who  are  responsible  for  the  allocation  of  resources  (6:40).  General 
Bernard  Randolph  insisted  that  he  was  out  of  the  loop  when  it  came  to 
program  decisions  and  that  the  streamlined  management  structure  was,  in 
fact,  working  (24:50).  He  saw  his  role  as  one  of  setting  up  the 
structure,  identifying  the  resources,  providing  test  facilities,  and 
providing  the  training  programs  that  managers  need  to  carry  out  their 
jobs  (24:51). 

In  September  1987,  the  Department  of  Defense  issued  DoD  Directive 
5000.43,  Acquisition  Streamlining.  Ibis  directive  has  made  streamlining 
an  official  policy.  The  overall  theme  of  the  directive  is  to  encourage 
minimizing  bureaucratic  program  requirements  which  do  not  add  value  to 
the  accomplishment  of  the  mission  without  abandoning  government 
oversight  outright  (8:2-4). 
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In  1989,  amid  growing  concern  that  the  results  of  the  Packard 
Commission  were  being  ignored  and  that  the  Goldwater-Nichols  Defense 
Reorganization  Act  of  1986  was  not  being  implemented,  President  George 
Bush  directed  the  Secretary  of  Defense  to  develop  "a  plan  to  improve  the 
defense  procurement  process  and  management  of  the  Pentagon"  (5:1). 

Secretary  of  Defense  Dick  Cheney  presented  his  plan  on  12  June  1989. 

This  far  reaching  plan  addressed  many  of  the  concerns  discussed  in  the 
previous  section  of  this  chapter.  While  formulating  his  plan,  the 
Secretary  had  the  goals  of  maintaining  the  strength  and  readiness  of  the 
Armed  Forces;  acquiring  needed  weapon  systems  at  lower  cost,  in  less 
time,  and  at  the  promised  performance;  encouraging  industry  and 
government  to  adhere  to  the  highest  standards  of  integrity  and 
performance;  and  promoting  greater  public  confidence  in  DoD’s  management 
(5:1). 

His  plan  called  for  bolstering  the  DoD  management  structure  by 
giving  the  Under  Secretary  of  Defense  for  Acquisition  (USD/A)  the 
authority  intended  by  the  Packard  Commission  (5:3).  If  the 
recommendations  are  implemented,  a  DoD  Executive  Cocmittee  will  be 
formed  which  will  serve  as  the  "senior  deliberative  and  decision  making 
body  within  DoD  for  all  major  defense  issues"  (5:5).  The  Defense 
Planning  and  Resources  Board  (DFRB)  will  manage  a  revitalized  Planning, 
Programming  and  Budgeting  System  (5:5).  A  major  area  of  emphasis  for 
the  DPRB  will  be  realistic  long-term  planning  (5:6).  The  DKffi  will  work 
closely  with  the  Defense  Acquisition  Board4  to  insure  that  major 
programs  are  adequately  planned  (5:7). 

♦The  Defense  Acquisition  Board  (DAB)  is  responsible  for  setting  policy  on 
the  acquisition  of  DoD  major  systems. 
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The  report  also  includes  detailed  plans  to  institute  clear  command 
channels,  enhance  program  stability,  limit  report  requirements, 
establish  small,  high-qvrrJ.ity  staffs,  enhance  conmunications  with  users, 
and  improve  system  development  (5:8).  These  plans  will  have  profound 
effects  on  current  management  practices  with  many  of  them  to  be 
implemented  in  the  near  term.  The  changes  reconmended  are  not  only 
limited  to  the  DoD  and  industry.  Several  concern  key  reforms  in 
Congress  (5:25).  The  intent  of  these  and  Air  Force  initiatives  along 
the  same  lines  is  to  provide  for  a  stronger  defense  while  eliminating 
inefficiencies  in  the  system. 

Conclusion 

The  1980 's  have  clearly  been  a  time  of  tunultuous  change  in  the 
defense  acquisition  business.  Many  changes  have  been  instituted  and 
many  more  are  planned.  The  DoD  and  its  military  branches  still  have  a 
ong  way  to  go,  as  evidenced  by  Secretary  Cheney’s  defense  management 
plan. 

This  literature  review  has  attempted  to  shed  c  me  light  on  the 
complex  issues  surrounding  defense  procurement  in  general  and 
acquisition  strategy  planning  in  particular.  The  literature  has  clearly 
illustrated  the  need  for  strategic  planning  of  acquisitions  by  the  DoD. 
Indeed,  it  has  been  found  that  "effective  long  range  strategic  planning 
in  the  major  systems  acquisition  process  results  in  successful  programs” 
(3:27). 

Unfortunately,  the  literature  also  illustrates  the  cumbersome 
strategy  review  process  implemented  by  Air  Force  Systems  Command  in  the 
form  of  the  ASP.  When  locked  at  in  isolation,  the  ASP  may  not  appear  to 
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be  too  much  of  a  burden  on  the  program  manager .  When  it  is  looked  at  in 

the  context  of  all  of  the  other  reviews  which  the  program  manager  must 

deal  with,  however,  that  burden  appears  to  grow. 

The  impact  is  that  more  resources  may  be  devoted  to  preparing  for 
and  attending  reviews  and  briefings,  defending  program  decisions, 
and  responding  to  queries  from  higher  levels  than  are  devoted  to 
developing  and  implementing  strategic  and  operational  plans. 

(3:45) 

It  is  the  responsibility  of  higher  headquarters  staffs  to  provide 
the  program  manager  with  the  assistance  he  requires,  not  burden  him  with 
cumbersome  review  processes.  The  goals  of  the  ASP  process  are  directed 
the  right  way.  The  implementation  of  the  ASP  itself,  however  is  more 
questionable.  Strides  have  been  made  recently  to  improve  the  process, 
but  there  appears  to  be  room  for  further  improvement. 

Secretary  of  Defense  Cheney  concluded  that  "the  American  people 
expect  those  who  nanage  the  nation’s  defense  effort  will  aim  high.  And 
they  deserve  nothing  less"  (5:27).  The  remainder  of  this  thesis  will 
assess  the  costs  and  benefits  of  the  ASP  process  to  the  program  manager 
and  make  recommendations  for  improvements. 
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Ill .  Methodology 


Introduction 

The  previous  chapter  described  the  formulation  of  acquisition 
strategies  in  general  and  the  acquisition  strategy  panel  (ASP)  process 
in  particular.  It  also  described  the  contemporary  calls  for 
streamlining  of  the  defense  acquisition  process  and  built  a  case  for 
examining  the  costs  and  benefits  associated  with  the  ASP  process  in  the 
hopes  of  formulating  a  recommendation  for  continuing,  discontinuing,  or 
improving  this  process.  This  chapter  describes  the  methodology  used  to 
arrive  at  those  recommendations . 

The  research  was  conducted  in  the  form  of  personal  interviews  with 
personnel  who  had  first-hand  experience  with  the  ASP  process.  It  began 
with  a  literature  review  of  guiding  regulations,  acquisition  strategy 
planning  fundamentals,  and  trends  in  acquisition  streamlining  and 
reform.  Next,  the  appropriate  population  was  determined  and  a  list  of 
personnel  to  be  interviewed  was  drawn  up.  Information  gained  during  the 
literature  review  was  used  to  formulate  an  interview  guide.  Interviews 
were  conducted  in  person  and  via  the  telephone  using  this  guide.  The 
data  generated  were  then  collected  and  analyzed  using  a  spreadsheet 
program.  Conclusions  and  recommendations  for  further  research  were 
drawn  from  these  results. 

Justification 

The  aim  of  this  thesis  was  to  isolate  one  variable  in  the  Air 

Force  acquisition  process,  specifically  the  use  of  the  ASP  to  develop 

program  acquisition  strategy,  and  determine  if  its  use  caused  a 

positive,  neutral  or  negative  impact  on  the  overall  success  of  a 
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program.  Ideally,  the  way  to  resolve  this  research  problem  would  be  to 
conduct  an  experiment  (15:60)  on  identical  programs,  varying  the  use  of 
the  Acquisition  Strategy  Panel  while  holding  all  other  variables 
constant.  The  results  of  the  programs  which  utilized  the  ASP  approach 
could  then  be  compared  to  the  results  of  those  programs  which  did  not 
use  the  ASP.  Any  differences  in  program  effectiveness  could  then  be 
linked  to  the  use  or  non  use  of  an  ASP. 

Unfortunately,  this  type  of  experiment  would  be  next  to  impossible 
to  accomplish  due  to  time,  money,  and  personnel  constraints  as  well  as 
the  volatility  of  the  program  management  process.  To  complicate  the 
situation  further,  historical  data  regarding  quantitative  benefits 
resulting  from  the  use  of  the  ASP  process  was  not  available. 

Accordingly,  an  ex  post  facto  design  was  deemed  most  appropriate 
(15:60);  the  ASP  would  have  to  be  judged  strictly  on  the  basis  of 
qualitative  data  in  the  form  of  opinions  held  by  personnel  involved  with 
the  process.  Because  of  the  long  acquisition  life-cycles  associated 
with  the  fielding  of  new  weapon  systems  and  the  relatively  recent  (1987) 
implementation  of  the  ASP  process,  and  because  the  success  of  an 
acquisition  program  can  only  be  measured  upon  its  completion,  it  would 
have  been  next  to  impossible  to  prove  causation  between  the  use  of  the 
ASP  and  the  success  of  a  program.  Therefore  the  research  was  conducted 
in  the  form  of  a  descriptive  study  based  upon  structured  interviews  of 
personnel  with  first-hand  knowledge  of  the  process.  The  interviews 
measured  their  perceptions  relative  to  the  ASP. 
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Instrument 


The  final  interview  guide  used  during  the  study  was  developed 
based  upon  the  findings  of  the  literature  review  and  the  results  of 
three  pilot  interviews  conducted  with  personnel  from  Aeronautical 
Systems  Division  at  Wright-Patterson  AFB,  Ohio.  An  introduction  Which 
stated  the  objectives  of  the  study  and  the  nature  of  the  interview  was 
provided  at  the  beginning  of  the  guide.  Qjestions  were  structured  to 
gather  as  much  quantitative  information  as  possible,  given  the 
qualitative  nature  of  the  study.  To  this  end,  Likert  opinion  scales 
were  used  throughout  the  interview  guide.  These  scales  were  of 
particular  use  since  the  goal  was  to  change  or  improve  a  process 
(15:256).  For  those  interviews  which  were  conducted  in  person,  the 
respondent  was  given  the  last  page  of  the  guide  which  solicited  further 
comments  that  were  not  discussed  during  the  interview  itself.  The 
interview  guide  is  included  in  Appendix  B. 

Structure  of  the  Questions.  Each  question  was  structured  in  one 
of  three  ways.  The  first  type  asked  for  specific  information.  Question 
4  serves  as  an  example  of  this  type.  It  asked  each  respondent  to 
specify  the  dollar  value  of  the  contract  discussed  at  the  ASP.  The 
second  type  quantified  the  respondents'  opinions  through  the  use  of 
five-point  Likert  opinion  scales.  In  each  of  these  cases,  the 
respondents  were  also  given  the  opportunity  to  embellish  upon  the 
specific  issue.  The  last  type  consisted  of  open  ended  questions  Which 
the  respondents  were  asked  to  answer  in  any  way  they  felt  was 
appropriate.  For  example,  question  23  asked  the  respondent  to  finish 
the  statement  "the  greatest  benefit  of  the  ASP  is:." 


The  first  type  of  question  was  used  where  specific  information  was 
required  from  the  respondent.  These  were  vised  when  the  goal  was  to  find 
demographic  information  or  assess  the  costs  and  benefits  associated  with 
the  ASP. 

The  second  type  was  used  when  the  opinions  of  the  respondents 
relative  to  a  particular  aspect  of  the  ASP  were  sought.  These  Likert 
scale  questions  asked  the  respondents  to  rate  their  reactions  to 
statements  relative  to  a  five-point  scale.  A  one  on  the  scale  indicated 
that  the  respondent  strongly  disagreed  with  the  statement.  A  two 
indicated  the  he  or  she  moderately  disagreed  with  the  statement.  If  the 
respondent  felt  neutral,  a  three  was  recorded  as  the  response.  Moderate 
or  strong  agreement  with  the  statement  was  recorded  as  a  four  or  five, 
respectively.  The  respondents  were  then  asked  if  they  wanted  to  add  any 
comments  to  their  responses. 

The  last  type,  the  open  ended  statements,  were  used  to  reduce  the 
effect  of  bias  introduced  by  the  interviewer.  The  open  ended  format 
were  considered  the  most  effective  —i  cases  of  high  sensitivity  (4:167). 
Accordingly,  these  were  reserved  for  the  most  sensitive  of  the  issues 
covered  by  the  guide.  In  each  case,  the  respondent  was  asked  to  respond 
in  any  manner  they  felt  appropriate,  to  include  a  non-response. 

Categories  of  Questions.  The  questions  used  in  the  guide  fell 
into  four  distinct  categories.  The  first  category  asked  the  respondents 
to  provide  demographic  information.  The  second  category  sought 
information  regarding  preparation  for  an  ASP  briefing.  The  next 
category  dealt  with  the  conduct  of  the  panel  briefing  itself.  The  last 
category  sought  information  on  general  impressions  regarding  the  overall 
ASP  process. 
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Questions  Concerning  Demographics.  Hie  first  four  questions 
of  the  guide  gathered  demographic  information.  The  relationship  of  the 
respondent  to  the  ASP  process  was  ascertained  in  question  1.  Question  2 
captured  the  time  frame  in  which  the  respondent  was  involved  with  the 
process.  The  first  part  of  the  question  was  geared  primarily  toward 
panel  member  respondents  since  their  involvement  would  have  been  over  a 
period  of  time  rather  then  at  a  specific  point  in  time.  The  second  part 
of  the  question  was  geared  toward  program  management  personnel.  Many 
changes  have  occurred  in  the  ASP  process  since  its  inception  in  1987. 
Therefore,  the  point  in  time  in  which  a  program  manager  was  involved  may 
have  had  an  affect  on  his  or  her  perception  of  the  process.  Also 
affecting  the  PM’s  perception  may  have  been  at  what  point  in  the 
program’s  life-cycle  the  ASP  occurred.  The  literature  pointed  out  that 
strategy  development  was  vital  in  the  early  stages  of  a  program.  The 
ASP,  however  is  conducted  repeatedly  over  a  program’s  life-cycle  and, 
thus,  a  valid  question  is  what  utility  the  ASP  has  in  the  latter  phases 
of  a  program’s  life. 

The  third  question  was  geared  specifically  at  program  managers  and 
simply  categorizes  the  programs  by  the  dollar  value  of  the  contracts 
under  consideration  during  the  ASP.  This  was  vised  to  determine  if 
program  size  affected  the  FM’s  perceptions  of  the  process.  The  last 
question  served  as  a  check  to  insure  that  the  only  ASPs  considered  in 
the  study  were  joint  ASPs  which  involved  Headquarters  AFSC  personnel. 
This  is  discussed  more  fully  in  the  "Population  and  Sample"  section  of 
this  chapter. 

Questions  Concerning  Preparation  and  Guidance.  In  the 
previous  chapter,  the  complexity  of  the  ASP  was  illustrated.  Appendix  A 
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shows  all  of  the  topics  which  the  PM  is  expected  to  cover  during  the 
briefing.  It  follows  that  the  process  of  preparing  for  the  ASP  could  be 
expected  to  be  a  large  task.  How  well  prepared  the  PM  was  for  an  ASP 
and  how  much  difficulty  he  or  she  had  in  the  preparation  process  could 
have  an  impact  on  the  success  and,  consequently,  his  or  her  attitude 
toward  the  ASP.  In  terms  of  costs  and  benefits,  the  costs  associated 
with  the  process  would  come  primarily  in  the  area  of  preparation  for  the 
ASP. 

The  first  of  the  preparation  questions  served  as  an  overview  and 
asked  the  respondent  if  he  or  she  agreed  that  preparation  for  the  ASP 
was  straightforward.  In  the  literature  review,  many  items  were  discussed 
which  were  produced  by  command  and  product  division  headquarters  to 
assist  the  program  manager  in  preparing  for  the  ASP.  The  second 
question  asked  the  respondent  if  they  agreed  that  each  of  these  items 
was  helpful  during  the  preparation  process.  Provision  was  made  for 
respondents  who  answered  that  they  did  not  use  the  item  or  were  unaware 
that  the  item  existed.  As  a  follow  up  question,  respondents  were  asked 
if  there  were  any  other  items  which  were  not  mentioned  that  were  of  use 
in  preparing  for  the  ASP. 

Question  4  asked  the  respondents  to  quantify  the  amount  of  effort 
that  went  into  preparation.  During  the  course  of  the  preliminary 
interviews  used  to  develop  the  guide,  a  program  manager  stated  that  the 
mere  act  of  preparing  for  the  ASP  served  to  alter  the  program’s  strategy 
and  solidify  the  program  office’s  position  relative  to  the  acquisition 
strategy.  As  a  result,  question  5  in  the  guide  asks  the  respondent  if 
he  or  she  agreed  with  the  statement  that  the  preparation  process 
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resulted  in  an  altered  program  strategy.  Question  6  asked  the 
respondent  for  open-ended  comments  on  the  preparation  process. 

This  section  was  geared  primarily  to  program  managers  and,  as 
such,  was  not  appropriate  for  the  panel  member  respondents. 

Accordingly,  panel  members  were  not  asked  to  respond  to  this  section. 
They  were,  however,  asked  how  much  and  what  types  of  preparation  they 
went  through  prior  to  each  ASP  they  participated  in.  Hie  results  of 
this  question  were  recorded  in  space  provided  at  the  and  of  the  guide. 

Questions  Concerning  the  Conduct  of  the  ASP.  This  section 
of  the  guide  asked  the  respondents  for  their  views  regarding  the  way  the 
ASP  was  actually  conducted.  Many  of  these  questions  got  to  the  heart  of 
the  research  objectives  discussed  in  Chapter  I. 

The  literature  review  pointed  to  the  fact  that  AFSC  was  concerned 
with  how  the  PMs  were  viewing  the  process.  One  concern  was  that 
teamwork  on  the  part  of  the  PMs  and  the  panel  members  was  vital  to  the 
overall  value  of  the  ASP.  Question  7  asked  the  respondents  if  they 
agreed  with  the  statement  that  the  panel  was  conducted  in  a  non- 
conf rontat i onal  way,  as  this  would  be  important  to  the  teamwork  aspiect 
of  the  process.  Question  8  followed  the  same  theme.  Since  the  command 
did  not  want  the  PMs  to  view  the  process  as  a  "square-filling"  exercise, 
respondents  were  asked  if  they  agreed  with  the  statement  that  the 
comments  received  from  the  panel  members  were  constructive. 

Question  9  dealt  directly  with  research  objective  nunber  1  by 
asking  the  respondents  if  they  agreed  that  the  ASP  redirected  overall 
program  strategy.  The  literature  review  also  highlighted  the  importance 
of  written  documentation  of  the  agreed  upon  acquisition  strategy.  The 
command  showed  its  concern  by  publishing  a  guide  on  writing  minutes. 


51 


Since  much  of  what  was  said  during  an  ASP  had  a  direct  impact  upon  the 
program's  overall  strategy,  close  coordination  between  the  written 
minutes  and  the  verbal  discussion  during  the  ASP  was  vital.  Question  10 
addressed  this  issue  by  asking  the  respondents  if  they  agreed  thrt  the 
written  minutes  paralleled  the  oral  comments  made  during  the  briefing. 

Research  objective  nvrnber  2,  regarding  the  implementation  of  ASP 
recommendations,  was  dealt  with  through  question  11  of  the  guide. 
Questions  12  and  13  asked  the  respondents  if  the  recommendations  of  the 
panel  had  positive  or  negative  effects  on  the  program  respectively  and, 
as  such,  resolved  research  objective  nianber  3.  Question  14  asked  the 
respondents  to  quantify  any  benefits  resulting  from  recommendations  of 
the  ASP.  This  information  could  then  be  compared  to  the  amount  of 
effort  expended  in  preparation  to  yield  a  cost/benefit  analysis  of  the 
ASP. 

One  portion  of  the  literature  review  referred  to  the  potential 
benefit  of  the  ASP  as  a  consensus  building  vehicle.  Question  15  asked 
the  respondents  if  they  agreed  that  the  ASP  built  a  consensus  of  support 
for  the  program.  The  pay-off  in  consensus  building  would  be  expected  to 
be  reduced  oversight  or  redirection  of  the  strategy  later  in  the 
program’s  life-cycle.  Question  15a  addressed  this  issue.  The 
literature  review  also  pointed  out  that  the  PM  may  have  had  little 
control  over  the  formulation  of  the  program’s  basic  acquisition 
strategy.  If  this  was  the  case,  the  value  to  the  PM  of  going  through  an 
ASP  would  be  expected  to  be  relatively  limited.  Question  16  asked  the 
respondents  if  they  agreed  that  the  strategy  was  developed  prior  to  the 
assignment  of  the  FM. 
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lestions  Concerning  Overall  Impressions.  The  last  series 
of  questions  focused  on  the  respondents  general  impressions  of  the  toted. 
ASP  process.  This  section  included  the  open-ended  questions  in  which 
the  respondents  were  asked  to  answer  in  any  way  they  felt  was  most 
appropriate. 

The  literature  review  showed  that  the  ASP  was  to  be  viewed  as  a 
tool  for  the  FM  to  use  in  the  development  of  the  program's  acquisition 
strategy.  The  usefulness  of  this  "tool"  then,  would  be  limited  if  the 
ASP  was  conducted  in  a  less  than  totally  professional  atmosphere. 
Question  17  asked  the  respondents  if  they  agreed  with  the  statement  that 
the  ASP  was  conducted  in  a  professional  manner  by  all  participants. 
Another  issue  that  was  uncovered  during  the  preliminary  interviews  was 
the  belief  that  the  ASP  did  not  have  to  be  conducted  at  the  Headquarters 
AFSC  level.  The  feeling  was  that  the  ASP  would  be  just  as  effective  if 
held  at  the  product  division  without  the  involvement  of  AFSC  panel 
members.  This  theme  served  as  the  motivation  for  questions  18  and  19. 
The  questions  asked  the  respondents  if  they  agreed  with  the  statements 
that  the  comments  of  product  division  panel  members  and  AFSC  panel 
members,  respectively,  were  constructive.  The  thought  here  was  that  if 
both  sets  of  panel  members  were  offering  constructive  conments  then  the 
argument  that  both  were  necessary  was  valid. 

Question  20  asked  the  respondents  to  rate  the  total  ASP  process  on 
a  scale  of  one  to  five  with  one  being  lowest  and  five  being  highest. 
Since  it  did  not  deal  with  any  one  specific  issue  of  the  aSP  process, 
this  question  served  to  sinmarize  the  respondents ’  feelings  toward  the 
ASP.  Low  opinions  concerning  the  issues  highlighted  in  previous 
questions  combined  with  a  high  average  score  on  this  question  would 
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argue  for  making  improvements  to  the  process.  Low  opinions  concerning 
the  detailed  issues  combined  with  a  low  average  score  on  this  question 
would  argue  for  discontinuing  the  process.  And  high  opinions  on  the 
issues  coupled  with  a  high  average  score  on  this  question  would  argue 
for  continuing  the  process  without  need  for  making  improvements. 

The  last  three  questions  were  open-ended.  Question  21  asked  the 
respondent  to  state  what  he  or  she  felt  was  the  greatest  benefit  of  the 
ASP.  Question  22  asked  the  respondent  to  state  what  he  or  she  felt  the 
greatest  problem  was  with  the  ASP.  And  question  23  asked  the  respondent 
how  the  ASP  process  could  be  improved.  In  each  case  the  respondents 
were  told  that  a  non  answer  would  be  appropriate  if  they  did  not  feel 
that  there  was  a  benefit,  a  problem,  or  a  way  or  need  to  improve  the 
ASP.  These  questions  were  included  primarily  to  allow  the  respondents 
to  comment  on  areas  which  may  have  been  neglected  by  the  author.  They 
gave  the  respondents  an  opportunity  to  express  themselves  in  ways  that 
were  not  allowed  elsewhere  in  the  interview  guide  due  to  the  structuring 
of  the  questions.  These  questions  also  tie  directly  to  the  research 
objectives  of  this  study,  namely  to  determine  the  value  (question  21)  of 
the  ASP  relative  to  its  costs  (question  22)  and  to  seek  areas  in  the 
process  needing  improvement  (question  23). 

Validity,  Reliability  and  Practicality  of  the  Instrument. 

The  "goodness"  of  a  research  instrument  is  measured  in  terms  of  its 
validity,  reliability,  and  practicality.  It  is  considered  valid  if  it 
actually  measures  what  it  was  supposed  to  measure.  Likewise,  it  is 
considered  reliable  if  the  measurement  procedure  is  accurate  and 
precise.  And,  it  is  considered  practical  if  it  is  economical, 
convenient,  and  easily  interpreted.  (15:94) 
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If  the  research  findings  can  be  generalized  across  persons, 
settings  or  times,  the  measurement  tool  is  said  to  be  externally  valid 
(15:94).  The  purpose  of  this  research  was  to  examine  a  very  specific 
aspect  of  the  Air  Force  acquisition  process  and,  as  such,  was  not 
intended  to  be  generalized  across  broader  populations  or  settings. 

Since  the  main  thrust  of  the  research  was  to  suggest  improvements  to  the 
ASP  process,  the  external  validity  of  the  research  instrument  was  not  an 
issue.  Therefore,  the  conclusions  and  recommendations  of  this  thesis 
only  apply  to  the  Air  Force’s  ASP  process.  Because  of  this,  inferences 
based  upon  the  findings  of  this  thesis  should  be  applied  to  broader 
populations  and  settings  only  after  very  careful  consideration. 

The  internal  validity  of  the  instrument  refers  to  "the  extent  to 
which  differences  found  with  a  measuring  tool  reflect  true  differences 
among  those  being  tested",  and  is  comprised  of  content,  criterion 
related,  and  construct  validity  (15:94).  Content  validity  of  the 
instrument  was  assured  in  three  ways.  The  first  of  these  was  the  depth 
of  the  literature  review  which  pointed  out  many  of  the  areas  which 
needed  to  be  considered  in  any  study  of  the  ASP  process.  Preliminary 
interviews  were  used  to  uncover  issues  that  were  not  brought  out  in  the 
literature  review.  And  finally,  each  respondent  was  asked  if  there  were 
any  other  issues  they  felt  were  important  which  were  not  covered  during 
the  interviews.  In  addition,  those  who  were  interviewed  in  person  were 
given  a  form  with  the  author’s  address  on  which  they  could  record  at  a 
later  date  any  further  thoughts  on  the  subject  which  they  felt  were 
relevant  to  the  study. 

The  question  of  the  instrument’s  criterion  related  validity  was  an 
elusive  one.  The  major  dilemma  in  conducting  this  research  was  that  it 


was  virtually  impossible  to  determine  the  effect  of  the  ASP  on  the 
degree  of  success  experienced  by  a  major  acquisition  program.  The 
author,  in  the  process  of  designing  the  research  instrument,  found  no 
sources  of  information  which  would  point  conclusively  to  the  effect  of 
the  ASP  on  program  success.  Without  such  information,  the  only  option 
open  was  to  perform  an  opinion  study  with  personnel  involved  in  the 
process.  However,  to  increase  the  criterion  related  validity  of  the 
instrument,  question  14  of  the  instrument  asked  the  respondents  to 
provide  quantitative  data  regarding  the  benefits  of  the  ASP.  While  this 
may  be  a  weak  area  of  the  study,  it  is  hoped  that  readers  of  this 
research  will  find  value  in  the  findings  when  applying  them  to  the 
current  ASP  proc . ss . 

Many  of  the  questions  in  the  instrument  were  interrelated  to 
ensure  construct  validity.  It  was  expected  that  respondents  answering 
positively  to  questions  8  and  12  or  negatively  to  question  13  would  also 
rate  the  overall  ASP  process  highly  in  question  20.  By  the  same  token, 
those  responding  negatively  to  questions  8  and  12  and  positively  to 
question  13  would  be  expected  to  rate  the  overall  process  poorly  in 
question  20. 

The  reliability  of  the  instrument  is  a  necessary,  but  not 
sufficient  condition  for  its  validity  (15:98).  Reliability  means  that 
the  instrument  is  free  of  random  or  unstable  error.  In  this  case, 
reliability  was  measured  in  terms  of  its  stability,  which  refers  to  its 
ability  to  elicit  the  same  response  from  the  same  respondent  with  the 
same  instrument.  This  was  done  by  adding  "Question  0"  to  the  guide. 
(Xiestion  0  was  placed  in  order  after  the  demographic  questions,  but 
before  any  of  the  other  questions  and  was  a  rephrasing  of  question  20. 
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The  degree  to  which  the  respective  answers  were  similar  served  as  an 
indicator  of  the  instrument * s  reliability.  This  approach  was  chosen 
because  the  ranks  and  positions  of  the  personnel  interviewed  made  it 
nearly  impossible  to  re- interview  them  several  weeks  later.  That 
translated  to  the  less  than  optimal  result  of  reliability  being  measured 
at  the  same  time  that  the  interview  was  conducted.  The  statistical 
analysis  used  to  test  for  stability  of  the  instrument  is  discussed  in 
the  "Statistical  Tests"  section  of  this  chapter. 

A  research  instrument  is  considered  practical  if  it  is  economical, 
convenient  and  interpretable  (15:100).  The  research,  conducted  in  the 
form  of  in-person  and  telephone  interviews,  was  economical  to  perform 
because  of  the  small  population  size  involved  (less  than  40 
individuals).  These  interviews  were  convenient  in  that  they  allowed  for 
direct  feedback  from  the  respondents  and  resulted  in  an  extremely  low 
non-response  rate  while  minimizing  the  difficulty  in  conducting  than. 

It  is  hoped  that  the  documentation  included  in  this  thesis  will  make  it 
readily  interpretable  for  interested  persons  in  the  future. 

Population  and  Sample 

As  was  pointed  out  in  the  "Scope  and  Limitations"  section  of 
Chapter  1,  this  thesis  was  concerned  with  the  impact  of  Joint  ASPs  on 
program  effectiveness.  Joint  ASPs  were  defined  in  Chapter  II  as  those 
involving  AFSC  and  product  division  panel  members  and  were  reserved  for 
major  or  special  interes  programs  as  designated  by  the  Secretary  of  the 
Air  Force.  Consequently,  the  population  of  interest  was  limited  to  Air 
Force  personnel  with  first-hand  knowledge  of  the  Joint  ASP  process.  The 
major  focus  was  on  program  managers  since  they  would  be  in  the  best 
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position  to  determine  the  effect  of  the  ASP  on  their  programs.  They 
also  would  have  a  perspective  of  the  ASP  in  terms  of  the  effort  that 
went  into  their  preparation.  It  was  expected  that  many  of  the  program 
managers  who  participated  in  ASPs  prior  to  1988  would  have  already  left 
the  service  or  moved  on  to  other  jobs.  Thus,  the  emphasis  was  placed  on 
interviewing  program  managers  who  had  participated  from  1988  on. 

To  get  a  balanced  view,  the  opinions  of  the  AFSC  panel  members 
were  considered  important  to  the  research.  Their  views  were  expected  to 
shed  some  light  on  objectives  and  benefits  of  the  ASP  which  were  not 
discovered  during  the  literature  review  and  which  may  not  have  been 
apparent  to  the  program  managers.  Their  views  were  compared  to  those  of 
the  program  managers  to  point  out  areas  of  misconmunication  and,  thus, 
were  very  useful  in  making  recommendations  for  improvements  to  the 
process.  Product  division  panel  members  were  not  singled  out  for 
interviews  because  of  the  focus  on  Conmand  involvement  in  the  ASP. 
Several  of  the  program  managers  interviewed,  however,  were  also  members 
of  their  product  divisions’  standing  ASP  panel  membership  and  spoke  from 
both  perspectives. 

Figure  3  in  Chapter  II  showed  that  the  total  number  of 
Headquarters  AFSC  ASPs  (or  Joint  ASPs)  conducted  up  to  that  point 
numbered  approximately  30  and  therefore,  the  maximum  number  of  program 
managers  that  would  be  interviewed  would  not  exceed  30.  Figure  5  in  the 
same  chapter  showed  that  there  were  approximately  11  standing  panel 
members  at  the  Headquarters  AFSC  level.  Turn  around  of  personnel  and 
organizational  changes  further  reduced  the  size  of  the  population.  This 
resulted  in  a  total  population  size  of  less  ther  *5  personnel.  With 
such  a  small  population,  random  sampling  was  next  to  impossible.  As  a 
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result,  it  was  decided  that  as  many  personnel  as  possible  would  be 
contacted  for  an  interview.  Appendix  C  lists  the  programs  involved  in 
the  Joint  ASP  process  from  which  contacts  were  attempted . 

Data  Collection  Plan 

The  small  population  involved  made  it  possible  for  all  of  the 
interviews  to  be  conducted  in-person  or  over  the  telephone .  Each 
interview  was  conducted  by  the  same  interviewer  using  the  same  interview 
guide  to  minimize  variations  in  question  phraseology.  The  interviews 
lasted  from  one-half  hour  to  one  hour  and  were  set  up  in  advance  so  that 
the  respondent  was  aware  of  the  subject  matter.  Each  interview  began 
with  an  explanation  of  the  objectives  of  the  research  and  the  types  of 
questions  that  would  be  asked.  The  respondents  were  asked  to  provide 
additional  information  for  each  of  the  questions  as  they  saw  fit.  At 
the  end  of  the  interviews,  the  respondents  were  asked  if  they  had  any 
other  comments  which  they  wanted  to  add  at  that  time.  In  addition, 
those  who  were  interviewed  in-person,  were  given  a  form  to  fill  out  and 
mail  if  any  other  ideas  regarding  the  ASP  occurred  to  them  after  the 
interview  was  concluded. 

The  goal  was  to  perform  as  many  in-person  interviews  as  possible 
since  this  would  allow  the  interviewer  to  gain  as  much  detail  and  depth 
from  the  respondent  as  possible  (15:160).  This  method  also  allowed  the 
interviewer  to  probe  for  additional  information  and  adjust  to  the  needs 
of  the  respondent  (15:161).  The  problems  with  this  type  of  interview  is 
its  cost  and  the  possible  introduction  of  bias  by  the  interviewer.  The 
problems  of  bias  were  discussed  earlier.  Cost  precluded  the  interviewer 
from  performing  all  of  the  interviews  in-person.  Program  managers  from 
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Aeronautical  Systems  Division  (ASD)  at  Wright-Pat terson  AFB,  Ohio,  were 
interviewed  in  person  because  of  the  division’s  proximity  to  the  Air 
Force  Institute  of  Technology.  In  addition,  two  Temporary  Duty  (TDY) 
trips  were  conducted  in  support  of  this  thesis.  The  first  trip  was  to 
Electronic  Systems  Division  (ESD)  at  Hanscom  AFB,  Massachusetts,  where 
ESD  personnel  were  interviewed.  The  second  trip  was  to  Air  Force 
Systems  Command  Headquarters  at  Andrews  AFB,  Maryland.  Ibis  trip 
allowed  for  face-to-face  interviews  with  the  AFSC  ASP  panel  members. 

The  balance  of  the  respondents  were  interviewed  over  the  phone. 
These  included  personnel  from  Munitions  Systems  Division  (MSD)  at  Eglin 
AFB,  Florida;  Human  Systems  Division  (HSD)  at  Brooks  AFB,  Texas;  Space 
Systems  Division  (SSD)  at  Los  Angeles  AFB,  California;  and  Ballistic 
Systems  Division  at  Norton  AFB,  California.  Telephone  interviews  are 
considered  almost  as  good  as  face-to-face  interviews  since  there  are  no 
meaningful  differences  in  response  bias  between  the  two  types  (4:13). 

The  greatest  benefit,  however,  is  this  method’s  reduced  cost  as  compared 
to  face-to-face  interviews  (15:169). 

Statistical  Tests 

The  majority  of  the  quantitative  data  gathered  in  this  research 
was  in  the  form  of  opinion  rankings  based  upon  a  five-point  Likert 
Scale.  This  data  is  considered  to  be  ordinal  level,  and  as  such  is 
limited  to  the  use  of  the  median  and  percentile  as  the  measures  of 
central  tendency  and  dispersion,  respectively  (15:90). 

The  stability  of  the  instrument  was  tested  by  comparing  the 
results  of  question  20  with  those  of  question  0  through  the  use  of  the 
"paired  t-test".  This  test  is  appropriate  when  there  is  only  one  set  of 
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individuals  and  two  observations  are  made  on  each  individual  (11:343). 
The  assumptions  required  to  carry  out  this  test  are  that  the  data 
consist  of  n  independently  selected  pairs  with  the  differences  in  the 
pairs,  D,  being  normally  distributed.  Hie  null  hypothesis,  Ho,  is  that 
the  mean  of  the  differences  is  equal  to  0,  and  the  alternate  hypothesis, 
Ha,  is  that  the  mean  is  not  equal  to  0,  indicating  that  the  pairs  of 
data  do  not  have  the  same  distribution  (11:344).  In  this  case,  a  two- 
tailed  test  is  appropriate  and  Ho  will  be  rejected  in  favor  of  Ha  if 

t-obs  ">.  or  ^obs  -  ~t*y2,n-l  (11:345)  (1) 

where  tobs  is  the  observed  t-value  and  t«/2(n_,  is  the  critical  t-value, 
which  was  found  using  Table  A. 5  of  Devore  (11:635).  The  observed  t- 
value  was  calculated  using  the  Statistix  II  software  package.  The 
results  of  the  stability  test  are  discussed  in  Chapter  4. 

Because  ordinal  level  data  was  used,  the  construct  validity  of  the 
instrument  was  evaluated  by  cross  tabulation  of  the  relevant  data.  The 
cross  tabulations  were  used  to  show  if  the  related  data  followed  the 
trends  hypothesized  earlier  in  this  chapter. 

Summary 

This  chapter  has  discussed  the  methodology  which  was  used  during 
the  course  of  this  research.  An  opinion  survey  of  personnel 
knowledgeable  in  the  ASP  process  was  conducted  through  face-to-face  and 
telephone  interviews  using  a  structured  interview  guide.  The  findings 
of  the  research  are  documented  in  the  next  chapter. 
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IV.  Findings 


Introduction 

The  previous  chapters  set  the  stage  for  the  research  which  was 
conducted  during  this  thesis.  The  interviews,  which  comprise  the  body 
of  data  for  this  thesis,  were  conducted  over  a  two  month  period.  The 
data  gathered  during  those  interviews  is  summarized  in  the  sections 
which  follow.  Each  of  the  interview  questions  will  be  discussed 
separately  with  the  quantitative  data  being  presented  first  followed  by 
a  summary  of  the  comments  received  on  each  question. 

Numerous  attempts  were  made  to  contact  specific  personnel  for  the 
interviews.  Many  of  these  personnel  had  moved  on  to  other  jobs  or  had 
retired  from  the  Air  Force.  A  total  of  29  interviews  were  conducted  and 
included  in  the  data  base.  Of  the  29,  9  were  conducted  over  the 
telephone  while  20  were  conducted  in  person.  Each  person  was  asked  the 
same  questions  but  allowed  to  embellish  in  any  words  which  they  thought 
were  appropriate. 

As  discussed  in  Chapter  III,  each  question  posed  in  the  interview 
guide  fell  into  one  of  three  structural  categories.  Several  of  the 
questions  were  posed  as  statements  of  opinion  with  the  respondents  being 
asked  to  state  their  level  of  agreement  with  the  statement.  The  levels 
were  recorded  on  a  five-point  Likert  scale  with  a  one  indicating  strong 
disagreement  with  the  statement,  a  three  indicating  neither  agreement 
nor  disagreement,  a  five  indicating  strong  agreement,  and  a  two  or  four 
indicating  moderate  disagreement  or  agreement,  respectively.  The 
results  of  questions  of  this  type  will  be  discussed  both  in  terms  of  the 
scale  and  the  respondents’  comments. 
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The  data  was  compiled  and  analyzed  using  a  spreadsheet  developed 
with  the  Quattro  software  package.  Hie  spreadsheet  information  is 
included  in  Appendix  D.  It  depicts  the  raw  data  and  the  calculated 
values  used  throughout  this  chapter.  Some  of  the  demographic 
information  is  not  included  in  the  appendix  to  guarantee  the  anonymity 
of  the  respondents. 

Reliability  of  the  Instrument.  As  discussed  in  Chapter  3,  a  test 
of  the  instrument's  stability  of  was  conducted.  This  was  done  by 
comparing  the  responses  of  two  similar  questions,  questions  20  and  0, 
through  the  use  of  a  paired  t-test.  Appendix  E  documents  this  test. 
Because  different  wording  was  used  in  each  question  to  evoke  two 
responses  to  the  same  issue,  and  because  one  question  was  asked  at  the 
beginning  of  the  interview  while  the  second  was  asked  at  the  end,  the 
data  pairs  were  considered  to  be  independent.  Likewise,  a  Wilke- 
Shapiro  test  of  the  paired  data  showed  that  the  distribution  of  the 
differences  was  found  to  approximately  follow  a  normal  distribution. 

The'  two  conditions  provide  the  basis  for  using  the  paired  t-test. 

Using  a  95%  confidence  level,  the  critical  t-value  was  found  to  be  2.056 
from  Table  A. 5  of  Devore  (11:635)  and  the  observed  t-value  was  found  to 
be  1.73.  Accordingly,  the  distributions  of  the  responses  to  the  two 
questions  were  found  to  be  equivalent  and,  thus,  the  instrument  was 
found  to  be  stable  and  reliable. 

Construct  Validity  of  the  Instrument.  In  Chapter  III,  it  was  also 
pointed  out  that  the  instrument's  construct  validity  could  be  tested  by 
cross-tabulating  the  responses  of  questions  8,  12,  and  13  with  the 
responses  to  question  20.  The  results  of  these  cross -tabulations  can  be 
seen  in  Appendix  F.  High  responses  to  questions  8  and  12  were  expected 
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to  result  in  high  responses  to  question  20.  Likewise,  low  responses  to 
questions  8  and  12  were  expected  to  coincide  with  low  responses  to 
question  20.  On  the  other  hand,  responses  to  question  13  were  expected 
to  inversely  coincide  with  responses  to  question  20.  Of  the  25 
respondents  giving  question  8  a  resonse  of  4  or  5,  20  responded  with  a  4 
or  5  to  question  20.  The  3  who  responded  to  question  8  with  a  3  or 
lower  gave  question  20  a  response  of  2  or  3.  With  regard  to  question 
12,  of  the  20  who  responded  with  a  4  or  5,  15  also  gave  question  20  a 
score  of  4  or  5.  Of  the  5  who  responded  with  a  3  or  less,  3  also  gave 
question  20  a  score  of  3  or  less.  Question  13  followed  the  opposite 
pattern.  Of  the  19  scoring  question  13  with  a  3  or  less,  18  gave 
question  20  a  score  of  3  or  higher.  The  results  follow  the  expectations 
for  high  construct  validity. 

Demographic  Information 

The  majority  of  the  personnel  interviewed  (18)  were  program 
managers  or  project  officers  on  programs  which  had  been  through  the  ASP 
process.  Three  of  those  interviewed  were  assigned  to  a  product  division 
ASP  secretariat  while  the  remaining  eight  were  Headquarters  AFSC  ASP 
panel  members.  The  interviews  included  eight  personnel  from 
Aeronautical  Systems  Division  (ASD)  at  Wright-Patterson  AFB,  Ohio;  one 
person  from  Ballistic  Systems  Division  (BSD)  at  Norton  AFB,  California; 
six  personnel  from  Electronic  Systems  Division  (ESD)  at  Hanscom  AFB, 
Massachusetts ;  1  person  from  Human  Systems  Division  (HSD)  at  Brooks  AFB, 
Texas;  two  personnel  from  Munitions  Systems  Division  (M3D)  at  Eglin  AFB, 
Florida;  and  three  personnel  from  Space  Systems  Division  (SSD)  at  Los 
Angeles  AFB,  California. 
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Those  interviewed  included  ten  civilian  employees  of  the  Air  Force 
and  19  military  personnel.  The  civilians  included  four  members  of  the 
Senior  Executive  Service  (SES)  with  the  remainder  ranging  from  GM-13  to 
GM-15.  The  military  personnel  included  two  brigadier  generals,  six 
colonels,  seven  lieutenant  colonels,  three  majors  and  one  captain.  The 
panel  members  had  served  on  the  AFSC  panel  an  average  of  2.25  years, 
with  five  having  been  on  the  panel  since  its  inception  in  1987. 

In  Chapter  2,  the  dollar  thresholds  required  for  a  Joint  ASP  were 
said  to  be  $200  million  in  research  and  development  (R&D)  funding  and/or 
$1  billion  in  production  funding  or  were  of  special  interest.  Of  the  18 
programs  included  in  the  interviews,  seven  fell  below  the  established 
thresholds,  implying  that  they  were  special  interest  programs.  The 
average  R&D  costs  of  all  18  programs  was  $340  million  and  the  average 
production  contract  was  for  $2,450  million.  The  literature  also 
stressed  the  importance  of  developing  acquisition  strategies  early  in  a 
program’s  life.  Only  three  of  the  eighteen  programs  involved  in  the 
study  were  in  the  concept  exploration  or  demonstration  validation  phases 
while  the  remaining  15  programs  were  in  full  scale  development  or 
production  at  the  time  of  their  respective  ASPs.  Figure  6  shows  the 
distribution  of  the  programs  by  acquisition  phase. 

Questions  Concerning  Preparation  and  Guidance 

The  questions  in  this  section  of  the  interview  guide  dealt  with 
the  program  manager’s  efforts  to  prepare  for  the  ASP.  They  attempted  to 
capture  the  amount  of  effort  expended  in  preparation,  the  relative  case 
of  preparation,  items  that  were  helpful  in  preparing  for  the  ASP,  and 
suggestions  for  improving  the  preparation  process.  Headquarters  AFSC 
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panel  members  were  not  asked  the  questions  in  this  part  of  the  guide, 
but  were  asked  what  forms  of  preparation  they  went  through  prior  to  an 
ASP.  Their  comments  are  discussed  toward  the  end  of  this  section. 

Question  1:  Preparation  for  the  ASP  was  Straightforward.  Question 
1  asked  the  respondent  if  he  or  she  agreed  with  the  statement  that 
preparation  for  the  ASP  was  straightforward.  In  some  cases,  the 
respondents  did  not  understand  what  was  meant  by  the  term 
"straightforward".  In  those  cases,  the  interviewer  explained  that  the 
term  meant  that  preparation,  while  possibly  requiring  a  lot  of  work,  did 
not  involve  confusion  or  false  starts.  Nearly  two  thirds  of  those 
responding  did  not  feel  that  preparation  for  the  ASP  was  a 
straightforward  process,  answering  with  a  strongly  disagree,  disagree, 
or  a  neutral.  Only  one  third  agreed  (24%)  or  strongly  agreed  (10%)  that 
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preparation  was  straightforward.  Figure  7  shows  the  breakout  of 
responses . 

Each  of  AFSC’s  product  divisions  has  different  infrastructures  to 
deal  with  assisting  their  program  managers  in  preparing  for  the  ASP.  Of 
interest  was  the  breakout  of  this  question  by  product  division.  Figure 
8  is  a  stacked  histogram  of  the  responses  to  this  question  broken  out  by 
product  division.  The  largest  grouping  of  ASD  program  managers  agreed 
that  preparation  was  straightforward  while  the  largest  grouping  of  other 
product  division  program  managers  was  in  the  disagree  category.  While 
this  may  imply  that  the  larger  product  divisions  have  made  it  easier  for 
their  program  managers  by  providing  more  resources  to  help  them  prepare, 
the  result  may  be  biased  by  the  fact  that  these  product  divisions  have 
more  personnel  with  ASP  experience  which  could  be  passed  on  to  other 
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program  managers  within  the  product  division. 

Questions  2  and  3:  Tools  to  Help  Prepare  for  the  ASP .  These  two 
questions  dealt  with  the  ASP  preparation  tools  that  were  available  to 
the  program  managers.  Question  2  listed  eight  preparation  tools  and 
asked  the  program  managers  if  each  item  was  helpful  in  preparing  for  the 
ASP.  Question  3  asked  the  program  managers  what  other  tools  they  used 
to  prepare  for  the  process. 

The  literature  review  pointed  out  several  tools  that  were 
available  to  the  program  manager  to  make  ASP  preparation  easier.  These 
included  AFSC  regulations,  product  division  regulations,  product 
division  handbooks  which  included  the  AFSC  ASP  Preparation  Guide,  a 
video  tape,  command  and  product  division  ASP  secretariats,  the  DSMC 
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Acquisition  Strategy  Guide,  and  an  ACSC  Acquisition  Strategy  Guide.  For 
each  of  these  items,  question  2  asked  the  program  managers  to  indicate 
their  level  of  agreement  with  the  statement  that  the  item  was  helpful  in 
preparing  for  the  ASP.  If  they  could  not  respond  on  a  particular  item, 
they  were  asked  if  they  had  known  about  the  item  but  chose  not  to  use  it 
or  if  they  were  simply  unaware  that  the  item  existed. 

Figure  9  ranks  the  tools  in  terms  of  the  average  responses  to  the 
questionaire  statement.  While  the  use  of  an  average  is  not,  strictly 
speaking,  statistically  "correct",  it  is  helpful  in  giving  some 
indication  of  the  relative  rankings  of  the  items.  Figure  10  shows  the 
percentage  of  respondents  who  did  not  use  or  were  unaware  of  the  item. 

It  should  be  noted  that  the  average  responses  depicted  in  Figure  9 
represent  the  opinions  of  those  respondents  who  actually  used  the  item. 
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Figure  10.  Percentage  of  Respondents  Who  Did  Not  Use  the  Tools 

The  item  that  appeared  most  helpful,  the  AFSC  Video  Tape,  was  only  used 
by  30%  of  the  program  managers.  However,  the  product  division  ASP 
secretariats,  which  were  ranked  second  in  terms  of  helpfulness,  were 
used  by  85%  of  the  program  managers.  Command  regulations  concerning  the 
ASP  were  ranked  third  and  were  used  by  75%  of  the  respondents.  Ranked 
fourth  in  terms  of  helpfulness  was  the  Headquarters  AFSC  ASP  secretariat 
which  was  used  by  75%  of  the  program  managers.  The  DSMC  Acquisition 
Strategy  Guide  was  ranked  fifth,  but  was  used  in  only  37.5%  of  the 
cases.  The  product  division  handbooks  were  ranked  sixth  and  used  in 
only  45%  of  the  cases.  In  this  case,  all  of  the  SSD  and  HSD  personnel 
interviewed  indicated  that  they  were  unaware  of  a  handbook,  implying 
that  those  two  product  divisions  may  not  have  had  a  handbook.  Product 
division  regulations  concerning  the  ASP  were  used  in  70%  of  the  cases, 
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but  were  ranked  7th  in  terras  of  helpfulness.  Finally,  the  .  jCSC 
Acquisition  Strategy  Guide  was  not  used  by  any  of  the  personnel 
interviewed.  It  should  be  noted  that  the  two  guides  (ACSC  and  DSMC)  did 
not  deal  specifically  with  preparing  for  an  ASP  but  served  as  a  guide  to 
preparing  program  strategy. 

Question  3  asked  the  program  managers  to  describe  any  other  items 
that  helped  them  prepare  for  their  respective  ASPs.  Six  program 
managers  stated  that  the  passing  on  of  experience  gained  by  other 
program  managers  who  had  been  through  the  ASP  process  was  very  helpful. 
The  lessons  learned  packages  published  by  AFSC  every  six  months  were 
singled  out  as  being  very  helpful  by  four  program  managers.  Direc* 
contact  with  command  panel  members  was  very  helpful  for  two  of  the 
program  managers.  Other  items  which  were  mentioned  by  one  or  more 
program  managers  were  examples  of  charts  used  in  *  'c.vious  ASPs  and  the 
experience  gained  in  developing  strategies  for  other  programs.  It 
should  be  noted  that  the  ASP  Preparation  Guide  published  by  the  command 
was  considered  helpful  and  was  included  as  part  of  the  product  division 
handbooks . 

Question  4:  Amount  of  Effort  Spent  Preparing  for  the  ASP.  All  of 
the  program  managers  were  asked  to  estimate  the  amount  of  labor  which 
went  into  preparing  for  the  ASP.  The  program  managers  were  asked  not  to 
include  the  effort  which  would  normally  be  used  to  develop  their 
programs’  acquisition  strategies.  All  of  the  respondents  had  to 
estimate  this  figure  as  specific  records  were  not  kept.  Many  had 
problems  breaking  out  ASP  preparation  labor  from  the  normal  effort  of 
developing  program  acquisition  strategy.  Accordingly,  the  figures 
obtained  varied  widely  from  program  to  program. 
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Figure  11.  ASP  Preparation  Time  -  in  Estimated  Man-Months 


Figure  1 1  shows  the  average  and  maximum  ASP  preparation  t  1- -aes  by 
product  division.  ASP  preparation  time  ran  from  a  minimum  of  0.25  man- 
months  to  a  maximum  of  30  man-months  with  an  average  of  9.13  man- 
months.  ESD  and  SSD  had  the  highest  averages  with  17.3  and  12.5  man- 
months,  respectively.  ASD  programs  showed  an  average  preparation  time 
of  7.54  man-months,  while  the  smaller  product  divisions  averaged  3.25 
man-months.  There  was  no  discemable  trend  in  preparation  time  as 
compared  to  program  size,  which  was  indicated  by  contract  value. 

Question  5:  ASP  Preparation  Resulted  in  an  Altered  Strategy.  This 
question  asked  the  respondent  to  state  his  or  her  level  of  agreement 
with  the  statement  that  preparation  for  the  ASP  resulted  in  an  altered 
program  strategy.  The  results  were  split  fairly  evenly  with  40% 
agreeing  or  strongly  agreeing  that  preparation  for  the  ASP  altered 
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Figure  12.  ASP  Preparation  Altered  Program  Strategy  -  Breakout  by 


Response 

program  strategy  while  45%  disagreed  or  strongly  disagreed.  The  two 
highest  scoring  categories  were  strongly  disagree  and  agree,  each  with 
35%  of  the  responses.  Therefore,  it  is  very  difficult  to  make  a  case 
either  way  with  regard  to  this  statement. 

Many  of  the  program  managers,  when  asked  to  comment  on  this 
question,  stated  that  the  majority  of  changes  which  occurred  during  the 
ASP  preparation  process  did  not  deal  directly  with  the  program  strategy, 
but  were  primarily  procedural  in  nature  or  minor  changes .  Several 
stated  that  the  formal  preparation  required  for  the  ASP  helped  to 
solidify  the  program  office  position  by  surfacing  disconnects  between 
functional  disciplines  and  the  program  office.  This  forced  the  parties 
to  resolve  the  problems  sooner  then  they  would  have  under  other 
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conditions.  Oie  stated  that  the  checklist  nature  of  ASP  preparation 
helped  fill  in  holes  in  the  strategy  while  another  stated  that  it  helped 
the  program  office  focus  on  the  risk  of  the  program.  And  another 
program  manager  stated  that  the  contract  type  for  the  program  was 
changed  because  of  issues  discovered  during  ASP  preparation. 

Question  6:  Other  Comments  on  Preparation.  This  open-ended 
question  asked  the  program  managers  if  they  had  any  other  comments  on 
ASP  preparation  or  suggestions  for  changing  the  preparation  process. 

Many  of  their  comments  dealt  with  the  structure  of  the  ASP  briefings. 
Several  of  the  program  managers  stated  that,  while  they  understood  that 
the  panel  members  needed  specific  information,  they  felt  that  the 
program  manager  should  have  had  more  flexibility  in  how  the  topics  were 
presented.  One  stated  that  the  focus  by  the  program  manager  should  have 
been  on  substance  over  form  while  another  stated  that  the  program 
manager  should  not  allow  the  regulation  to  capture  him,  but  rather,  the 
program  manager  should  capture  the  intent  of  the  regulation.  Several 
stated  that  the  briefing  should  be  scaled  and  tailored  to  the  specific 
program.  One  suggested  that  a  tutorial  regarding  the  specifics  of  the 
program  be  given  to  the  panel  members  in  advance  of  the  briefing  so  that 
the  ASP  could  focus  directly  on  strategic  issues. 

The  headquarters  panel  members  were  also  asked  this  question. 

Many  of  their  comments  fell  along  similar  lines.  The  Panel  Chairperson, 
Mrs  Druyun,  stated  that  one  of  the  hardest  items  to  assess  in  an  ASP  was 
a  program’s  risk.  At  the  time  of  this  writing,  the  command  was  looking 
at  ways  to  incorporate  a  NASA  risk  model  to  help  the  program  managers 
develop  consistent  risk  assessments.  Along  this  line,  another  panel 
member  stated  that  the  programmatic  and  technical  risk  must  be 
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understood  as  well  as  the  consequences  of  failure  in  a  particular  area. 
This  would  allow  for  the  early  anticipation  of  problems  and  help  the 
panel  members  focus  on  the  most  critical  areas.  Three  of  the  panel 
members  stated  that  open  communication  was  critical  to  the  success  of 
the  ASP  since  this  would  allow  for  the  discussion  of  the  critical 
program  issues.  Finally,  several  program  managers  and  panel  members 
stated  that  the  best  way  for  a  program  manager  to  prepare  for  an  ASP  was 
to  sit  through  one  several  months  prior  to  his  or  her  ASP. 

While  the  preparation  task  seemed  focused  on  the  program  office 
staff,  the  panel  members  also  prepared  for  each  ASP.  Several  program 
managers  and  most  of  the  panel  members  stated  that  adequate  preparation 
by  the  panel  members  was  as  vital  to  the  success  of  a  particular  ASP  as 
the  preparation  done  by  the  program  office.  Panel  member  preparation 
for  each  ASP  involved  gathering  as  much  information  regarding  the 
program  as  possible.  This  included  pre-briefings  by  project  officers; 
reviews  of  program  guidance,  documentat i on  and  reports;  and  the 
evaluation  of  contractor  history  and  performance.  A  problem  which 
faced  the  headquarters  panel  members  was  that  the  narrowing  role  of  AFSC 
was  making  it  harder  to  obtain  information  and  thus,  preparation  for  the 
ASPs  was  becoming  much  more  difficult.  One  panel  member  sunned  up  the 
importance  of  preparation  by  stating  that  the  worst  thing  for  a  panel 
member  to  do  was  to  walk  into  an  ASP  "cold." 

This  section  has  focused  on  preparation  for  the  ASP  with  regard  to 
the  amount  of  effort  expanded  and  the  ease  of  preparation.  Both  program 
managers  and  panel  members  emphasized  the  importance  of  adequate 
preparation  by  stating  that  thorough  preparation  resulted  in  the  most 
successful  ASPs  in  terms  of  value  to  the  program  manager.  The  next 
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section  focuses  on  what  happens  after  the  preparation  process,  namely 
the  conduct  of  the  ASP  briefing  itself. 

Questions  Concerning  the  Conduct  of  the  ASP 

This  part  of  the  interview  guide  concentrated  on  the  way  in  which 
the  ASP  was  conducted.  Also  of  interest  were  the  issues  of 
implementation  of  ASP  recommendations  and  the  impact  of  those 
recommendat i ons  on  the  programs.  Panel  members  as  well  as  program 
managers  were  asked  the  questions  in  this  section. 

Question  7:  The  ASP  was  Conducted  in  a  Non-Confrontational  Way. 
Respondents  were  asked  to  state  their  level  of  agreement  with  the 
statement  "the  ASP  was  conducted  in  a  non-confrontational  way" .  The 
results  are  illustrated  in  Figure  13.  Only  three  percent  of  the 
respondents  were  neutral  on  this  issue.  The  majority  of  the  respondents 
(82%)  strongly  agreed  or  agreed  that  the  ASP  was  non-confrontational. 
Only  14%  disagreed  with  the  statement  while  3%  strongly  disagreed. 

Several  program  managers  stated  that  there  was  a  small  degree  of 
confrontation  but  that  this  was  controlled  quickly.  One  program  manager 
stated  that  while  there  was  some  confrontation,  he  felt  that  it  was 
constructive.  Another  felt  that  panel  members  were  too  politically 
oriented  and  were  non-committal.  And  finally,  a  program  manager  stated 
that  some  of  the  advice  from  certain  panel  members  was  outside  of  their 
specialty  and  that,  in  some  cases,  certain  panel  members  became 
antagonistic. 

Panel  member  respondents  also  commented  on  this  issue.  Oie  stated 
that  about  two  thirds  of  the  panel  briefings  were  non-confrontational 
while  one  third  were  confrontational.  Ibis  panel  member  felt  that  the 
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Figure  13.  ASP  Was  Non-Confrontational  -  Breakout  by  Response 

confrontation  was  due  to  the  fact  that  the  panel  was  not  "wedded"  to  the 
programs,  whereas  the  program  managers  may  have  been  more  attached  to 
them.  He  also  stated  that  the  panel  could  feel  insulted  if  the  program 
offices  tried  too  hard  to  educate  the  panel  members.  Another  panel 
member  stated  that  the  issue  of  confrontation  was  very  "personality 
driven"  on  both  sides  of  the  table.  He  went  on  to  state  that  the  panel 
members  tended  to  be  dedicated  to  their  specialties  and  that  if  a 
program  manager  did  not  address  a  member’s  particular  area  of  interest, 
the  "functional  zealot"  (panel  member)  would  voice  dissatisf action.  One 
stated  that  the  ASP  was  "as  confrontational  as  a  program  manager  wanted 
to  make  it".  One  characterized  the  ASPs  as  starting  "somewhat 
confrontational"  followed  by  a  lessening  of  tensions  as  they  progressed. 
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And  finally,  another  panel  member  stated  that  the  ASP  was  a  "waste  of 
time"  if  it  was  confrontational. 

The  ASP  Chairperson,  Mrs  Druyun,  stated  that  the  issue  of 
confrontation  was  a  high  priority  of  hers.  She  felt  that  teamwork  on 
the  part  of  the  panel  members  and  the  program  office  personnel  was  key 
to  the  value  of  the  ASP.  Mrs  Druyun  explained  that  the  panel  members 
did  not  "have  a  monopoly  on  ideas"  and  that  their  role  was  to  offer 
advice,  not  confrontation.  She  believed  that  the  attitudes  of  the  co¬ 
chairpersons  were  crucial  in  ensuring  that  the  ASPs  did  not  become 
conf  rontat ional . 
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estion  8:  Panel  Member  Comments  Were  Constructive.  Again,  this 


issue  was  posed  as  a  statement  to  which  the  respondents  were  to  indicate 


Figure  14.  Comnents  Received  frcm  Panel  Members  Were  Constructive  - 

Breakout  by  Response 

their  levels  of  agreement .  Figure  14  depicts  the  breakout  of  responses. 
Two-thirds  of  the  respondents  agreed  that  the  comnents  of  panel  members 
were  constructive.  Nearly  a  quarter  of  the  respondents  strongly  agreed 
with  the  statement  while  the  remaining  10%  were  neutral.  None  of  the 
respondents  disagreed  or  strongly  disagreed  with  the  statement. 

Figure  15  shows  the  histogram  of  responses  to  question  8  as  broken 
out  by  product  division.  Hie  majority  of  the  program  managers, 
represented  by  the  product  division  blocks,  agreed  with  the  statement 
while  the  majority  of  panel  members  strongly  agreed  with  the  statement. 
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Figure  15.  Comments  Received  from  Panel  Members  Were  constructive 

Breakout  by  Product  Division 


This  indicates  a  slight  difference  in  perception  between  the  program 
managers  and  the  panel  members.  None  of  the  ASD  program  managers 
strongly  agreed  with  the  statement  but  three  of  the  smaller  product 
division  program  managers  did. 

Six  of  the  program  managers  commented  that  the  majority  of  the 
comments  received  from  panel  members  were  constructive,  but  that  some  of 
the  comments  were  not.  One  described  the  non-cons  true  tive  comments  as 
"fliers"  while  another  stated  that  there  were  always  some  "straphanger 
conments".  Another  reported  that,  while  the  majority  of  the  comments 
were  very  helpful,  one  of  the  panel  members  made  concluding  comments 
that  were  "off-the-wall".  Three  program  managers  reported  that  many  of 
the  comments  were  educational  in  nature,  in  that  the  panel  members 
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sought  more  information  on  the  specifics  of  the  programs  rather  than 
their  strategies.  One  of  these  program  managers  characterized  the 
comments  as  "mundane".  Finally,  one  program  manager  stated  that  the 
comments  served  as  a  good  "sanity  check"  and  that  if  the  program  manager 
could  not  counter  the  comments,  he  or  she  probably  hadn't  prepared 
adequately  for  the  briefing. 

The  comments  of  panel  members  regarding  this  issue  were  similar  to 
those  of  the  program  managers.  Most  felt  that  the  majority  of  the 
comments  by  panel  members  were  constructive,  but  that  there  were  some 
which  were  not.  Two  panel  members  stated  that  some  of  the  comments 
"were  not  focused"  or  were  "off  the  track."  One  stated  that  about  50% 
of  the  comments  were  constructive.  He  reported  that  some  of  the  panel 
members  did  not  say  anything  during  the  course  of  an  ASP  while  others 
were  known  for  "championing  causes."  Another  reported  that  while  the 
panel  members  always  "intended  the  comments  to  be  constructive" , 
sometimes  they  fell  short  due  to  a  "lack  of  total  insight  into  the 
program."  The  panel  chairperson  stated  that  most  of  the  comments  were 
very  constructive,  but  that  some  were  "not  always  as  constructive  as  I’d 
like." 

Question  9:  The  ASP  Redirected  Program  Strategy.  The  respondents 
were  given  the  statement  "the  ASP  redirected  overall  program  strategy" 
and  asked  to  indicate  their  levels  of  agreement  with  it  on  the  five- 
point  scale.  Figure  16  illustrates  the  breakout  of  responses.  None  of 
the  respondents  were  neutral  on  this  issue.  An  overwhelming  82%  of  the 
respondents  either  disagreed  or  strongly  disagreed  with  the  statement. 
The  remaining  18%  agreed  or  strongly  agreed  with  the  statement. 
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Figure  16.  ASP  Redirected  Program  Strategy  -  Breakout  by  Response 

It  was  interesting  to  note  that  six  out  of  eight  of  the  panel 
members  were  quick  to  point  out  that  the  ASP  was  advisory  only  and  that 
it  gave  recommendations,  not  redirection.  The  program  managers,  on  the 
other  hand,  were  not  as  concerned  with  the  terminology  of  the  statement . 
Many  stated  that  the  ASP  did  redirect  or  refine  minor  elements  of  the 
strategy  but  did  not  overturn  the  overall  strategy.  One  program  manager 
stated  that  there  was  "no  question"  that  the  ASP  redirected  his 
program’s  strategy  but  that  the  redirection  was  positive.  Several 
commented  on  the  types  of  recommendations  that  were  received.  These 
included  recommendations  on  the  terms  and  conditions  of  the  contract, 
the  fee  structure,  procedural  issues  on  contractor  teaming,  and 
scheduling  for  the  Defense  Acquisition  Board  (DAB).  Two  program 
managers  reported  that  recommendations  which  affected  outside 


organizations  were  not  adopted  by  those  organizations  and  thus,  the  ASP 
recommendations  were  non-binding.  One  panel  member  stated  that  the  ASP 
redirected  a  program’s  strategy  only  if  it  was  "basically  flawed."  The 
ASP  chairperson  stated  that  the  strategy  would  be  redirected  only  if  it 
violated  statutory  requirements. 

Question  10:  The  Effectiveness  of  the  ASP  Minutes.  The 
respondents  were  asked  to  express  their  level  of  agreement  with  the 
statement  "written  minutes  of  the  ASP  briefing  paralleled  oral  comnents 
made  during  the  briefing".  Figure  17  depicts  the  response  to  the 
statement.  Only  11%  were  neutral  or  disagreed  with  the  statement.  The 
remaining  89%  either  agreed  or  strongly  agreed  with  it.  The  respondents 
were  also  asked  to  informally  comment  on  the  timeliness  of  the  minutes. 
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Figure  17.  Minutes  Corresponded  to  Verbal  Comments  -  Breakout  by 

Response 

Seven  program  managers  made  positive  comments  about  the  minutes. 
They  stated  that  the  minutes  were  timely,  contained  no  surprises  and 
were  well  coordinated.  Oie  of  these  stated  that  the  minutes  were 
"crafted  in  a  way  that  would  not  change  any  major  tenets  of  the 
strategy" .  Four  program  managers  made  negative  comments .  Two  of  these 
reported  that  there  were  "some  minor  problems"  and  that  the  minutes  were 
"not  as  detailed"  as  the  comments,  while  two  stated  that  the  minutes 
were  not  timely. 

Five  of  the  panel  members  made  positive  comments  regarding  the 
minutes.  Oie  panel  member  stated  that  the  minutes  were  "a  good  review 
process"  while  another  stated  that  the  write-ups  of  action  items  were 
particularly  good  but  that  the  descriptive  items  were  a  little  less  so. 


While  three  scored  the  statement  highly,  they  also  pointed  out  some 
problems  with  the  minutes.  One  stated  that  the  inflections  and 
intensity  of  the  comments  could  not  be  captured  by  the  minutes.  Another 
stated  that,  since  the  minute  takers  were  typically  procurement 
specialists,  they  did  not  know  how  to  take  shorthand  which  resulted  in 
some  misquotes.  The  panel  chairperson  stated  that,  in  an  attempt  to 
ensure  that  the  minutes  accurately  reflected  the  comments  made  at  the 
briefing,  draft  minutes  were  sent  to  all  of  the  panel  members  and  field 
personnel  for  coordination. 


Question  11:  Implementation  of  ASP  Recommendations .  The 
respondents  were  asked  to  express  their  levels  of  agreement  with  the 
statement  "direction  received  from  the  ASP  has  been  implemented".  As 
indicated  in  the  discussion  of  question  9,  several  respondents  had 
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problems  with  the  use  of  the  word  "direction".  In  those  cases,  the 
interviewer  substituted  "direction"  with  "recomnendation"  in  the 
statement.  None  of  the  respondents  disagreed  or  strongly  disagreed  with 
the  statement.  Those  who  agreed  or  strongly  agreed  made  up  85%  of  the 
responses  while  the  remaining  15%  were  neutral.  Figure  18  depicts  the 
breakout  by  response. 
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Figure  19.  ASP  Direction  Was  Implemented  -  Breakout  by  Product 

Division 


Figure  19  illustrates  the  histogram  of  responses  to  question  11 
broken  out  by  product  division.  Hie  majority  of  program  managers 
strongly  agreed  with  this  statement  while  the  majority  of  the  panel 
members  agreed.  This  indicates  a  difference  in  perception  between  the 
program  managers  and  the  panel  members.  ASD  program  managers’  responses 
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were  more  closely  grouped  with  the  strongly  agree  category  than  were  the 
responses  of  the  smaller  product  division  program  managers. 

One  program  manager  stated  that  while  the  direction  was 
implemented,  the  program  ended  up  not  being  funded  by  the  user  and  thus, 
the  program  was  not  continued.  In  this  case,  it  was  interesting  to  note 
that  the  user  attended  the  ASP  and  expressed  support  for  the  program  but 
later  decided  not  to  fund  it.  Several  program  managers  stated  that 
there  was  some  "give  and  take"  on  the  implementation  of  recommendations 
and  that  they  did  not  feel  they  had  to  implement  the  recommendations  if 
they  had  strong  reservations  about  them. 

The  panel  members  felt  that  the  practice  of  closing  action  items 
ensured  that  the  recommendations  were  implemented  or  "rationalized 
away" .  One  panel  member  stated  that  the  recommendations  were 
"implemented  by  the  guys  who  needed  to"  and  that  they  were  acted  upon  in 
good  faith.  Another  felt  that,  while  the  reconmendations  were  not 
explicitly  said  to  be  binding,  the  positions  and  expertise  of  the  panel 
members  served  to  make  them  implicitly  binding. 

Questions  12  and  13:  The  Impact  of  ASP  Reconmendations.  In 
question  12  the  respondents  were  asked  to  express  their  levels  of 
agreement  with  the  statement  "if  ASP  reconmendations  were  implemented, 
some  or  all  had  a  positive  impact  on  the  program"  while  question  13 
asked  than  to  do  the  same  with  the  statement  "if  ASP  reconmendations 
were  implemented,  some  or  all  had  a  negative  impact  on  the  program". 
Figure  20  depicts  the  distribution  of  responses  to  these  questions.  The 
majority  of  responses  to  question  12  fell  in  the  agree  category  while 
the  majority  of  responses  to  question  13  fell  in  the  strongly  disagree 
category. 
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The  general  consensus  was  that  the  recomnendations  which  were 
implemented  had  a  positive  impact.  None  of  the  respondents  disagreed  or 
strongly  disagreed  with  the  statement  in  question  12.  Five  were 
neutral,  while  15  agreed  and  6  strongly  agreed.  Four  respondents  agreed 
or  strongly  agreed  with  the  statement  in  question  13,  while  three  were 
neutral,  three  disagreed  and  14  strongly  disagreed.  There  was  not  much 
variation  between  program  managers  and  panel  members,  nor  among  product 
divisions . 

Those  responding  favorably  to  question  12  were  asked  what  the 
positive  impacts  were  to  their  programs.  Two  program  managers  stated 
th»t  the  "weight  of  the  ASP"  resulted  in  SPO  direction  carrying  more 
priority  with  the  contractors.  One  stated  that  the  ASP  chairperson 
stood  behind  the  SPO  desire  tc  reduce  paperwork  and  that  her  support 
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eliminated  any  lateral  fights  between  the  program  office  and  functional 
offices.  Another  stated  that  the  ASP  recommended  deleting  a  requirement 
for  work  measurement  standards  on  the  contract,  saving  an  unknown  amount 
of  money  and  work.  Yet  another  stated  that  the  ASP  shed  some  light  on 
ways  to  mitigate  risk  due  to  a  tight  schedule.  Two  program  managers 
stated  that  it  was  difficult  to  assess  the  benefits  of  the  ASP.  One 
stated  that  he  would  not  have  implemented  the  recommendations  if  they 
did  not  have  a  perceived  positive  benefit.  And  finally,  a  program 
manager  reported  that,  even  after  his  program’s  ASP,  DoD  turned  over 
control  of  the  program  to  another  DoD  organization. 

Panel  members  consistently  reported  that  it  was  difficult  to 
assess  the  impact  of  their  recommendations  upon  a  program,  positive  or 
negative.  Many  stated  that  their  intent  was  to  make  positive 
recommendations  and  that  the  issue  of  negative  impacts  had  to  be  looked 
at  in  terms  of  time  horizons.  On  a  short  term  basis,  they  felt  that 
their  recommendations  could  have  a  negative  impact  due  to  the  increased 
workload  of  implementing  them,  but  that  in  the  long  term,  the 
recommendations  should  have  a  positive  impact. 

Question  14:  Quantitative  Benefits  Resulting  from  the  ASP.  Hie 
program  managers  were  asked  to  quantify,  if  possible,  the  benefits  they 
experienced  due  to  the  implementation  of  ASP  recommendations.  Only  one 
out  of  18  program  managers  was  able  to  quantify  a  benefit  and  even  this 
case  was  difficult  to  prove.  In  his  case,  the  original  strategy  called 
for  a  dual-source  production  effort.  The  ASP  recommended ,  based  on  the 
past  performance  of  the  single  contractor  through  FSD,  that  the 
production  contract  be  awarded  single-source.  Ibis  resulted  in  a  $10 
million  savings  which  would  have  otherwise  been  required  to  bring  a 
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second  source  on-line.  However,  potential  savings  based  on  improved 
pricing  due  to  a  second  source  were  not  considered  in  the  savings 
figure. 

Most  program  managers  reported  that  there  were  benefits,  but  that 
they  were  not  easily  quantifiable.  The  benefits  came  in  the  areas  of 
making  the  FMs’  jobs  easier  by  passing  along  lessons  learned,  and  the 
political  aspect  of  consolidating  the  Air  Force  position  on  the 
programs.  One  program  manager  reported  that  the  ASP  recommendation  to 
request  structural  assessment  data  ahead  of  time  resulted  in  less  time 
being  spent  in  the  source  selection  process,  and  that  another  suggestion 
avoided  a  "premature  dismantling  of  the  competitive  field".  One 
reported  that  the  ASP  helped  sell  a  multi-year  funding  proposal  to 
outside  organizations.  One  reported  that  the  ASP  helped  to  streamline 
the  program's  paperwork  trail.  Another  program  manager  reported  that 
the  ASP  approval  of  a  key  change  in  strategy  eliminated  the  need  for  FMD 
updates,  expediting  implementation  of  the  change.  And  finally,  a 
program  manager  stated  that  a  non-quantifiable  benefit  of  the  ASP  was 
that  it  served  as  a  venue  to  "try  out  ideas  in  front  of  smart  people". 

Question  15:  The  Consensus  Building  Aspect  of  the  ASP.  This  issue 
involved  two  statements.  In  the  first,  the  respondents  were  asked  to 
express  their  levels  of  agreement  with  the  statement  "the  ASP  process 
served  to  build  a  consensus  of  support  for  the  program".  Those  who 
agreed  with  this  statement  were  then  asked  to  express  their  level  of 
agreement  with  the  statement  "this  consensus  of  support  served  to  reduce 
oversight  or  redirection  later  in  the  acquisition  life-cycle".  Figure 
21  depicts  the  response  to  the  first  statement,  while  Figure  22  does  the 
same  for  the  second  statement. 
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Figure  21.  ASP  Built  a  Consensus  of  Support  -  Breakout  by  Response 

For  the  first  statement,  80%  agreed  or  strongly  agreed  that  the 
ASP  built  a  consensus  of  support  for  the  program.  Four  percent  were 
neutral  and  16%  disagreed  or  strongly  disagreed  with  the  statement.  On 
the  second  statement,  which  was  only  used  for  respondents  who  answered 
with  a  three,  four,  or  five  on  the  first  statement ,  62%  agreed  or 
strongly  agreed,  while  38%  disagreed  or  strongly  disagreed  with  the 
statement.  None  of  the  respondents  were  neutral  with  regard  to  the 
second  statement. 

On  the  positive  side  of  the  consensus  building  issue,  one  program 
manager  stated  that  this  was  "the  biggest  plus  in  the  whole  thing  (the 
ASP  process ) .  Another  reported  that  it  was  "easy  to  keep  trucking 
along”  once  the  consensus  was  built  and  that  the  high  level  of 
involvement  by  the  using  command  was  beneficial.  Oie  stated  that  the 
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Figure  22.  Consensus  Resulted  in  Less  Oversight  -  Breakout  by  Response 

consensus  was  of  use  in  seme  areas,  but  that  the  ASP  could  not  ensure 
stability  at  all  levels.  Others  reported  that  the  ASP  "built  a 
consensus  for  the  paperwork,  but  nothing  more"  and  that  there  was  no 
"consistency  on  issues  that  could  be  delegated".  Several  commented 
that,  while  the  ASP  did  serve  to  build  a  consensus  of  support  for  the 
program,  the  majority  of  the  oversight  and  redirection  experienced  by  a 
program  came  from  outside  of  AFSC  and,  accordingly,  the  consensus  built 
by  the  ASP  could  not  affect  the  amount  of  redirection  or  oversight. 

Two  of  the  panel  members  equated  the  term  "consensus  building" 
with  advocacy  for  a  program  which  they  emphatically  stated  was  not  a 
role  of  the  command  headquarters.  Four  of  the  panel  members  were  in 
agreement  with  the  program  managers  who  stated  that  the  ASP  built  a 
consensus  but  that  the  consensus  had  little  to  do  with  redirection  or 
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oversight  of  the  program.  According  to  the  panel  chairperson,  the 
consensus  building  aspect  was  an  important  part  of  the  ASP  process.  She 
stated  that  the  ASP  "facilitated  the  paperwork  flow"  and  that  the 
attendance  of  the  user  and  the  program  element  monitor  (FEM)  could  help 
build  a  consensus  that  went  beyond  the  headquarters. 


Figure  23.  Strategy  Developed  before  FM  Assigned  -  Breakout  by 


Response 

Question  16:  Program  Manager  Control  of  the  Strategy.  This 
question  asked  the  respondents  to  express  their  levels  of  agreement  with 
the  statement  "the  basic  program  strategy  was  developed  prior  to  the 
assignment  of  the  program  manager."  Figure  23  depicts  the  response  to 
this  statement.  The  results  were  mixed  with  45%  agreeing  or  strongly 
agreeing  with  the  statement  and  50%  disagreeing  or  strongly  disagreeing. 
The  remaining  5%  were  neutral.  All  three  of  the  program  managers  with 
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programs  in  the  early  stages  of  the  acquisition  life-cycle  (concept 
exploration  and  demonstration/validation)  agreed  or  strongly  agreed  with 
the  statement,  indicating  that  they  felt  they  had  a  little  control  over 
their  strategies.  In  four  out  of  six  FSD  programs  however,  the  program 
managers  answered  with  a  strongly  disagree,  indicating  that  they  felt 
they  did  have  control  over  their  strategies.  This  attitude  changed  for 
program  managers  with  programs  going  into  production.  Three  out  of  four 
agreed  with  the  statement. 

Four  program  managers  reported  that,  while  they  had  some  broad 
guidelines  as  to  what  the  strategy  should  have  been,  they  had  a  lot  of 
flexibility  with  the  strategy.  Oie  stated  that  the  basic  framework  was 
laid  out,  but  that  he  had  flexibility  in  the  implementation  of  the 
strategy.  Another  stated  that  there  was  a  lot  of  flexibility,  but  that 
often  the  program  manager  was  "locked  in  by  inertia".  On  the  other 
hand,  four  stated  that  the  program  manager  had  only  very  limited  control 
over  the  program’s  strategy.  One  of  these  program  managers  stated  that 
a  program  manager  was  lucky  if  "he  controlled  10%  of  the  strategy"  while 
another  felt  a  program  manager  had  control  of  about  20%  of  the  strategy. 
And  most  of  these  program  managers  felt  that  their  control  was  more  in 
the  area  of  their  strategys ’  detailed  implementation.  Another  stated 
that  a  program  manager  was  "duty  bound"  to  change  it  if  he  or  she  did 
not  agree  with  it.  The  conments  of  panel  members  paralleled  those  of 
the  program  managers. 

Questions  Concerning  Overall  Impressions 

The  previous  section  concerned  itself  with  the  actual  conduct  of 
the  ASP.  This  section  deals  with  the  total  ASP  process.  The  first  four 
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questions  of  this  section  follow  the  opinion  statement  structure  while 
the  last  three  are  open-ended. 


Figure  24.  ASP  Was  Conducted  Professionally  -  Breakout  by  Response 


Question  17:  Professionalism  of  ASP  Participants .  The  respondents 
were  asked  to  express  their  degrees  of  agreement  with  the  statement  "the 
ASP  was  conducted  in  a  professional  manner  by  all  participants" .  The 
respondents  overwhelmingly  agreed  with  the  statement,  with  92%  agreeing 
or  strongly  agreeing.  Of  these,  74%  strongly  agreed  with  the  statement. 
Only  eight  percent  were  neutral  or  disagreed  and  none  of  the  respondents 
strongly  disagreed.  Figure  24  depicts  the  breakout  of  responses  to  this 
statement . 

One  program  manager  noted  that  seme  of  the  participants  did  not 
get  what  they  wanted  and  were  visibly  upset  while  another  seated  that 
there  was  seme  functional  infighting  among  the  product  division 
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participants.  One  of  the  panel  members  stated  that  there  were  instances 
where  "the  arrogance  of  field  program  managers  came  through" .  It  should 
be  noted  however  that  most  of  the  respondents  stated  that  a  lack  of 
professionalism  was  not  a  problem.  One  panel  member  stated  that  even 
when  the  briefings  became  somewhat  confrontational,  they  were  conducted 
professionally.  And  another  panel  member  sunned  it  up  by  saying  "I 
haven’t  seen  a  fist  fight  yet." 

Questions  18  and  19:  The  Value  of  Product  Division  and  AFSC 
Comments.  Question  18  asked  the  respondents  to  express  their  degrees  of 
agreement  with  the  statement  "the  recommendations  of  product  division 
panel  members  were  of  value  to  the  program  manager",  while  question  19 
asked  them  for  their  levels  of  agreement  with  the  statement  "the 
recommendations  of  AFSC  panel  members  were  of  value". 

As  a  group,  the  respondents  reported  that  both  the  product 
division  and  the  AFSC  panel  member  reconmendations  were  of  value,  but 
that  the  AFSC  recommendations  were  of  slightly  more  value  than  those  of 
the  product  divisions.  Program  managers  scored  the  product  division 
recommendations  slightly  lower  than  they  scored  the  AFSC 
recommendations.  AFSC  panel  members  followed  the  same  trend,  giving 
their  remarks  a  slightly  higher  score  than  those  of  the  product 
divisions.  ASD  program  managers  felt  the  product  division 
recommendations  were  of  more  value  than  those  of  AFSC.  ESD  program 
managers  considered  the  recommendations  of  both  groups  to  have  the  same 
value.  The  other  product  division  program  managers,  however,  scored  the 
AFSC  reconmendations  higher  then  those  of  the  product  division. 

The  respondents  were  then  asked  to  comment  on  whether  the  ASP 
needed  to  be  made  up  of  AFSC  and  product  division  personnel  or  if  it 
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could  be  conducted  without  one  of  those  parties .  Ten  of  sixteen  program 
managers  that  commented  on  this  issue  argued  for  the  continued 
involvement  of  both  the  command  and  the  product  division.  Most  of  these 
contended  that  the  two  groups  brought  specific  benefits  to  the  ASP  that 
the  other  group  could  not.  Among  the  benefits  of  AFSC  involvement, 
according  to  these  program  managers,  were  the  independence  of  the  AFSC 
panel  members  from  the  program  which,  in  turn,  gave  them  a  more 
objective  viewpoint,  and  their  broader  view  of  a  program.  The  product 
division  members  were  considered  to  have  more  experience  with  the 
intricacies  of  the  type  of  weapon  system  being  discussed.  Three  out  of 
the  16  program  managers  commenting  on  this  issue  stated  that  the  ASP 
could  be  held  just  as  effectively  at  the  product  division  level.  And 
the  remaining  three  stated  that  the  ASP  could  be  conducted  without 
product  division  involvement. 

Only  one  of  the  AFSC  panel  members  interviewed  felt  that  the  ASP 
could  be  conducted  just  as  effectively  at  the  product  division  level. 

The  rest  all  argued  for  continued  AFSC  involvement  in  the  process. 

Among  the  benefits  that  they  cited  were  their  ability  to  apply  lessons 
learned  across  the  product  divisions,  their  proximity  to  the  political 
thinking  in  Washington  as  well  as  the  Air  Staff  and  Office  of  the 
Secretary  of  the  Air  Force,  their  broad  experience,  and  their  lack  of 
parochialism  with  respect  to  specific  programs.  Many  of  the  panel 
members  stated  that  the  synergism  possible  with  conmand  panel  membership 
was  greater  than  that  of  product  division-only  membership. 

Question  20:  Rate  the  Overall  ASP  Process.  This  question  asked 
the  respondents  to  rate  the  total  ASP  process  on  a  scale  of  one  to  five, 
with  a  one  being  the  worst  possible  rating  and  a  five  being  the  best 
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possible  rating.  Figure  25  illustrates  the  frequency  distribution  of 
the  responses.  Only  one  of  the  respondents  gave  the  ASP  a  rating  of  two 
or  lower,  while  22  of  the  respondents  gave  it  a  four  or  five.  The 
remaining  five  respondents  gave  the  ASP  a  three.  Table  1  depicts  the 
responses  to  this  question  as  cross -tabulated  with  the  program  phase  in 
which  the  ASP  occurred.  While  it  is  difficult  to  evaluate  the  ratings 
of  the  two  respondents  in  the  concept  exploration  and 

demonstration/validation  phases,  it  is  interesting  to  note  that  four  out 
of  seven  FSD  program  managers  rated  the  ASP  with  a  four.  Along  the  same 
lines,  five  out  of  eight  of  the  production  program  managers  rated  the 
process  with  a  four. 
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Table  1 :  Cross  Tabulation  of  Question  20  by  Program  Phase 

" . . . . .  — — ; 

|  |  ASP  Process  Rating  I  I 

|  Program  Phase!  1  2  3  4  5  j  Row  Total  j 

l - - - - 1— - 1 - 1 

|  Concept  Exp  |  0  0  0  0  1  j  1  | 

IDem/Val  |  0  0  0  1  0  |  1  | 

|  FSD  |  0  0  3  4  0  |  7  | 

I  Production  j  0  1  1  5  1  j  8  I 

h - - - j - | - , 

I  Column  Total  |  0  1  4  10  2  |  17  | 

I— . . . . . . i . . .  . . . ■■■„— . -  .1 . . I 

Question  21:  The  Greatest  Benefit  of  the  ASP.  The  respondents 
were  asked  to  finish  the  open  ended  statement  "the  greatest  benefit  of 
the  ASP  was  .  .  ."in  any  manner  they  felt  appropriate.  Only  one 
program  manager  felt  that  there  was  no  benefit  derived  from  the  ASP  and 
that  the  ASP  was  not  worth  the  effort.  All  other  program  managers  felt 
that  there  was  some  benefit  to  the  ASP.  Five  of  the  program  managers 
stated  that  the  greatest  benefit  was  that  the  ASP  served  as  a  "sanity 
check"  which  legitimized  the  strategy  planning  process.  The  panel,  by 
being  somewhat  removed  from  the  program,  was  able  to  view  the  strategy 
from  a  more  objective  viewpoint  then  program  office  personnel. 

Another  five  stated  that  the  greatest  benefit  was  that  the  ASP 
served  to  "bring  a  wide  spectrum  of  managers  together"  to  consolidate 
the  program’s  strategy.  It  forced  a  "walk-through"  of  the  decision  tree 
which  led  to  the  particular  strategy  selected  for  a  program  and  it 
forced  the  program  manager  "to  get  his  act  together"  across  all  of  the 
functional  disciplines.  Two  program  managers  stated  that  it  forced 
discipline  into  the  acquisition  strategy  planning  process.  And  three 
program  managers  reported  that  the  lessons  learned  from  other  programs 
brought  to  light  by  the  panel  members  were  the  greatest  benefit  of  the 
ASP. 
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Five  of  the  panel  members  stated  that  they  felt  the  greatest 
benefit  was  the  "real-time  application  of  lessons  learned  across  the 
command.  Another  panel  member  stated  that  the  synergistic  affect  of 
agreement  on  the  initial  strategy  was  the  greatest  benefit.  One  felt 
that  its  greatest  benefit  was  that  it  forced  a  resolution  of  the  users’ 
requirements  for  the  systems.  And  another  stated  that  it  allowed  the 
program  manager  to  refocus  his  or  her  efforts,  if  required,  earlier  than 
would  be  possible  otherwise.  A  panel  member  added  that  "any  problem 
resolved  in  the  planning  stages  makes  a  ten-fold  savings  in  effort 
expended  if  left  until  later  in  the  program".  He  stated  that  this  was  a 
significant  return  on  investment  for  the  effort  required  to  prepare  for 
the  panel. 

Question  22:  The  Greatest  Problem  with  the  ASP.  This  question 
asked  the  respondents  to  finish  the  open-ended  statement  "the  greatest 
problem  with  the  ASP  is  .  .  ."in  any  manner  that  they  felt  appropriate. 
One  program  manager  felt  there  were  no  major  problems  with  the  ASP. 

Five  program  managers  stated  that  the  greatest  problem  was  with  the 
scheduling  of  the  briefings  because  of  the  high  levels  of  the  personnel 
involved.  Four  felt  that  the  greatest  problem  was  the  amount  of 
preparation  required  for  the  briefings,  with  one  stating  that  the  level 
of  the  briefing  led  product  division  personnel  outside  of  the  program 
office  to  "go  bonkers"  with  briefing  requirements  that  AFSC  did  not 
necessarily  need. 

Three  stated  that  the  greatest  problem  was  in  the  amount  of 
information  required  in  the  briefings.  One  of  these  program  mangers 
felt  that  much  of  the  information  was  not  relevant  to  the  program's 
strategy,  while  two  felt  that  the  presentation  format  for  the  briefings 
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weis  too  rigid.  Two  program  managers  stated  that  a  lack  of  in-depth 
understanding  with  regard  to  the  specifics  of  their  programs  on  the  pert 
of  some  of  the  panel  members  reduced  the  value  of  the  ASP.  And  smother 
complained  that  the  panel  failed  to  "sustain  and  establish  an  AFSC 
support  position"  for  the  program  following  the  approval  of  its 
acquisition  plan. 

One  panel  member  felt  that  there  were  no  major  problems  with  the 
ASP.  Two  stated  that  the  biggest  problem  was  with  program  managers  who 
were  less  then  fully  open.  They  felt  that  this  could  lead  to  the 
approval  of  a  strategy  which  may  not  be  the  best  for  a  particular 
program.  One  echoed  the  concerns  of  program  managers  who  felt  that 
preparation  time  was  the  biggest  problem.  Another  stated  that  the 
advisory  nature  of  the  ASP  could  lead  panel  members  to  feel  that  they 
were  not  "stake-holders"  in  a  program.  One  felt  that  inadequate 
preparation  by  panel  members  was  a  problem  which  resulted  in  their  "not 
understanding  the  particular  program  well  enough" .  The  panel 
chairperson  stated  that  the  biggest  problem  with  the  ASP  weis  the  lack  of 
a  common  definition  of  risk  which  could  lead  to  inconsistent  decisions 
being  made.  And  another  panel  member  felt  the  major  problem  was  that 
the  ASP  currently  treats  all  programs  the  same,  regardless  of 
acquisition  phase,  while  the  ASPs’  utility  diminished  for  programs  in 
the  latter  stages  of  the  acquisition  life-cycle. 

Question  23:  Improvements  to  the  ASP  Process.  This  question  Eisked 
the  respondents  to  finish  the  statement  "the  ASP  could  be  improved  by  . 

.  ."in  any  manner  they  felt  appropriate.  Five  of  the  program  managers 
commented  that  the  make  up  and  preparation  of  the  panel  were  areas  which 
could  be  improved.  One  suggested  that  the  members  be  picked  for  their 
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ability  to  make  positive  contributions  to  the  strategy  planning  process, 
while  two  stated  that  the  panel  members  needed  to  be  better  prepared  for 
each  ASP.  One  commented  that  there  were  too  many  panel  members  and 
another  felt  that  the  process  was  "demeaned"  when  substitutes  attended 
the  briefings  in  place  of  the  appointed  panel  members.  He  further 
stated  that  ASP  participation  should  be  considered  a  primary  duty  of  the 
panel  member,  not  an  additional  duty. 

Four  program  managers  felt  that  the  process  could  be  improved  by 
allowing  for  greater  flexibility  in  the  content  and  format  of  the 
briefings.  One  of  these  program  managers  stated  that  the  interactive 
nature  of  the  ASP  needed  to  take  precedence  over  the  formal  presentation 
aspect,  while  another  felt  that  ASP  preparation  and  guidance  needed  to 
be  kept  current  with  the  latest  developments  in  acquisition  regulations 
and  guidance. 

Two  program  managers  felt  that  the  criteria  for  holding  an  ASP  at 
the  headquarters  AFSC  level  should  be  based  upon  the  range  of  decisions 
available  relative  to  a  program's  strategy  with  one  stating  that  the 
process  needed  to  be  de-emphasized  or  delegated  for  programs  going  into 
production  and  discontinued  altogether  for  programs  in  follow-on 
production  contracts.  One  program  manager  stated  that  all  ASPs  should 
be  delegated  to  the  product  division  while  splitting  it  into  two 
sessions.  One  session  would  identify  the  issues  facing  a  program 
relative  to  its  strategy  while  the  second  would  address  those  issues. 
Another  program  manager  felt  that  responsibility  for  the  ASP  should  be 
passed  on  to  the  program  executive  officer  (PEO)  which  would  result  in 
placing  approval  of  a  program's  strategy  at  the  same  level  as  its 
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decision  making.  And  finally,  a  program  manager  suggested  the  use  of 
video  tele-conferencing  to  conduct  the  ASPs. 

Two  panel  members  felt  that  providing  more  preparation  time  for 
all  panel  members  would  improve  the  process.  Two  panel  members  stated 
that  the  issue  of  adequate  risk  assessments  needed  to  be  addressed. 
Another  panel  member  felt  that  the  ASP  should  be  used  on  an  ad  hoc  basis 
for  programs  with  specific  problems ,  as  requested  by  the  program 
manager,  while  one  stated  that  the  ASP  should  be  emphasized  for  programs 
going  through  milestones  0,  1,  and  2  and  de-emphasized  for  all  other 
programs.  And  finally,  another  panel  member  felt  the  process  could  be 
improved  through  greater  openness  by  all  parties  and  a  freer  exchange  of 
information. 

Other  Comments 

Several  respondents  made  other  comments  during  the  course  of  the 
interviews.  Two  respondents  felt  that  the  key  to  a  good  ASP  was  not  the 
quality  of  the  briefing  charts,  but  the  make-up  of  the  ASP  team. 

Another  stated  that  the  ASP  was  necessary  as  a  vehicle  to  bring  AFSC 
corporate  wisdom  to  bear  and  that  its  importance  lay  in  the  early  stages 
of  a  program  where  the  consequences  of  poor  decisions  would  be  greatest. 
He  also  stated  that  ASPs  were  important  for  less  than  major  programs 
because  of  the  relative  inexperience  of  those  program  managers.  One 
respondent  felt  that  program  managers  would  be  best  served  by  stopping 
and  asking  questions  during  the  briefing  rather  then  letting  "the  charts 
run  the  briefing”. 

Another  respondent  felt  that  the  ASP  was  too  political  and  that  it 
should  make  independent  recommendations .  One  stated  that  the  ASP  was 
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not  a  problem,  but  that  coordination  of  the  acquisition  plan  was.  He 
felt  that  there  was  much  more  value  in  briefing  the  strategy  through  the 
ASP  then  coordinating  it  through  the  bureaucracy  in  the  acquisition 
plan.  And  one  respondent,  commenting  on  the  acquisition  process  in 
general,  argued  that  there  needed  to  be  a  better  mechanism  for 
forwarding  changes  in  acquisition  guidance  to  the  program  managers. 
Finally,  many  of  the  program  managers  felt  that  AFSC’s  implementation  of 
a  Joint  ASP  was  extremely  beneficial  and  should  not  be  reversed. 

Summary 

This  chapter  has  summarized  the  results  of  the  interviews 
conducted  for  this  thesis.  These  results  have  shown  that  the  ASP  is 
adding  value  for  the  program  manager.  However,  there  appears  to  be  room 
for  improvement.  Conclusions  based  upon  the  findings  discussed  in  this 
chapter,  recommendations  for  improvement  of  the  ASP  process,  and 
recommendations  for  further  research  are  discussed  in  the  final  chapter 
of  this  thesis. 
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V.  Conclusions  and  Reconraendations 

Introduction 

Chapter  II  of  this  thesis  built  a  case  for  examining  the  ASP 
process.  Chapter  III  discussed  the  methodology  which  was  used  to 
conduct  the  research  and  Chapter  IV  discussed  the  results  of  that 
research.  Chapter  I  stated  the  goals  of  the  research,  namely  to 
determine  if  the  ASP  process  was  adding  value  for  the  program  managers 
charged  with  developing  and  fielding  weapon  systems  for  the  Air  Force, 
and  to  suggest  ways  to  improve  the  process.  This  chapter  ties  together 
the  discussions  of  Chapters  II,  III,  and  IV  to  satisfy  the  objectives 
listed  in  Chapter  I. 

Conclusions 

This  section  discusses  the  implications  of  the  research  findings 
with  regard  to  the  research  objectives,  the  research  problem  statement , 
ASP  preparation  and  conduct,  and  the  overall  ASP  process.  The  next 
section  will  discuss  methods  to  improve  the  process. 

Research  Objective  1 .  The  first  research  objective  was  to 
determine  how  often  program  acquisition  strategies  were  redirected  by 
the  ASP.  The  research  showed  that  acquisition  strategies  generally  were 
not  redirected  as  a  result  of  the  ASP  process.  The  ASP  did,  however, 
make  recommendations  which  typically  resulted  in  minor  changes  to 
overall  acquisition  strategy.  Many  of  the  respondents  stated  that  the 
major  benefit  of  the  ASP  was  not  that  it  redirected  strategy,  but 
rather,  that  it  served  as  a  sanity  check.  This  translated  into 
contractor  and  other  outside  personnel  placing  more  credibility  in  a 
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program’s  strategy  because  it  had  been  "blessed"  by  the  AFSC  panel  of 
experts . 

Several  panel  members  worried  about  the  use  of  the  term 
"redirection"  and  pointed  out  that  the  ASP  "advised"  program  managers 
only.  This  preoccupation  with  the  terminology  obscured  the  basic  fact 
that  the  ranks  and  positions  of  the  panel  members,  as  well  as  their 
roles  with  regard  to  other  aspects  of  the  programs,  served  to  make  their 
recommendations  extremely  difficult  to  ignore.  Many  program  managers 
stated  that  they  felt  they  could  argue  with  what  they  considered  to  be 
flawed  panel  recommendations .  However,  the  panel  members  need  to 
understand  the  import  of  the  comments  they  make  at  ASP  briefings.  While 
they  may  feel  that  the  "advisory"  nature  of  the  ASP  may  shield  them  from 
responsibility  for  their  own  statements,  their  statements  do,  in  fact, 
carry  an  implicit  authority  and  weight  which  the  program  managers  can 
not  ignore. 

This  research  objective  also  sought  to  determine  the  quality  of 
the  recommendations  received  by  the  ASP.  The  findings  showed  that  the 
respondents  overwhelmingly  felt  the  comments  received  during  the  ASP 
were  constructive  when  looked  at  in  aggregate.  Several  respondents, 
however,  indicated  that  a  few  of  the  c Moments  were  not  constructive  at 
all.  And  there  were  indications  from  panel  members  that  there  were 
certain  panel  members  who  were  known  for  either  not  saying  anything 
during  an  ASP  or  for  consistently  making  "off-the-wall"  comments. 

Research  Objective  2.  The  second  research  objective  was  to 
determine  if  the  recommendations  made  by  the  ASP  were  being  implemented 
or  if  they  were  being  ignored.  The  research  showed  that  the 
recommendations  were  indeed  being  implemented.  None  of  the  respondents 
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felt  that  the  recommendations  were  not  being  implemented.  The  fact  that 
the  majority  of  the  program  managers  agreed  that  the  recommendations 
were  being  implemented  suggests  that  the  ASP  is  not  strictly  advisory 
and  that,  implicitly,  it  is  directive  in  nature. 

Research  Objective  3.  The  third  research  objective  was  to 
determine  whether  the  recommendations  that  were  implemented  had  a 
positive  or  negative  impact  on  the  programs.  The  findings  showed  that 
the  recommendations  which  were  implemented  did  have  positive  impacts  on 
the  programs.  This  was  illustrated  by  the  fact  that  the  majority  of 
respondents  "agreed"  that  the  recommendations  had  positive  impacts  and 
"strongly  disagreed"  that  the  recommendations  had  negative  impacts. 

Many  of  these  positive  impacts,  however,  did  not  deal  directly  with 
strategic  issues,  but  came  in  the  form  of  increased  program  office 
credibility  with  contractors  and  functional  offices  as  a  result  of  the 
strategies  having  been  supported  by  the  expert  panel. 

As  discussed  earlier  in  this  thesis,  a  primary  problem  in 
conducting  this  study  was  that  there  was  no  baseline  for  comparison  to 
determine  the  effects  of  ASP  recommendations .  The  findings  of  question 
14  illustrated  this  shortcoming.  Only  one  of  the  program  manager 
respondents  was  able  to  identify  quantitative  impacts  of  the  ASP  and 
even  this  one  was  questionable.  Therefore,  the  topic  was  dealt  with  in 
terms  of  the  program  managers*  and  panel  members’  feelings  on  the 
subject.  The  responses  regarding  impacts  also  had  to  be  couched  in 
terms  of  time  horizons.  As  might  be  expected,  implementation  of  the 
recommendations  could  easily  result  in  negative  impacts  in  the  short 
term,  yet  yield  positive  impacts  in  the  long  term,  which  many  program 
managers  felt  to  be  the  case. 
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Responses  to  questions  15  and  15A  of  the  interview  guide  showed 
that  the  ASP  was  also  useful  in  building  consensus  for  programs,  but 
that  this  consensus  did  not  necessarily  lead  to  a  reduction  of  oversight 
or  redirection  of  the  programs  later  in  their  life-cycles.  The 
consensus  building  aspect  of  the  ASP  could,  however,  be  considered  a 
positive  impact  on  the  program  since  it  did  typically  reduce  the  amount 
and  the  coordination  time  of  paperwork.  User  participation  in  the  ASP 
was  considered  to  have  the  positive  impact  of  giving  both  the  users  and 
the  program  managers  a  greater  understanding  of  each  others  requirements 
and  objectives  for  the  programs. 

Research  Objective  4.  The  fourth  research  objective  was  to 
determine  what  types  of  strategic  planning  help  the  program  managers 
were  getting  and  to  determine  what  type  of  help  they  needed.  The 
literature  review  contained  in  Chapter  II  satisfies  these  objectives. 
Clearly,  the  ASP  is  one  form  of  assistance  which  the  program  managers 
are  receiving.  The  ASP  falls  in  line  with  the  recommendations  of 
Bissett  and  Kerzner  that  personnel  responsible  for  acquisition  strategy 
development  be  given  access  to  senior  level  personnel  responsible  for 
acquisition  policy.  Among  the  other  forms  of  assistance  available  to 
program  managers  were  the  DSMC  and  ACSC  Acquisition  Strategy  Guides. 

AFSC  and  product  division  personnel  also  provided  FMs  with  assistance  in 
developing  strategies  and  ASP  briefings.  Regulatory  guidance  was  also 
available  on  the  subject.  However,  the  key  ingredient,  according  to  the 
literature  review,  was  top  level  involvement  in  strategy  development. 

Research  Problem  Statement.  The  ultimate  goal  of  this  research 
was  to  determine  if  the  ASP  had  an  impact  on  program  effectiveness. 
Because  of  the  lack  of  "hard"  data  on  the  subject,  this  was  measured 
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qualitatively.  Responses  to  question  20  of  the  interview  guide  showed 
that  22  out  of  28  respondents  rated  the  process  highly.  Of  the 
remaining  six,  five  were  neutral  and  only  one  gave  the  ASP  a  rating 
below  neutral.  The  results,  then,  indicate  that  the  ASP  does  indeed  add 
value  for  the  program  managers.  These  responses  were  given  in  the 
context  of  weighing  all  aspects  of  the  ASP,  including  the  amount  of  time 
and  effort  that  went  into  their  preparation. 

Demographics .  A  large  portion  of  the  programs  involved  in  this 
study  were  in  the  production  phase  of  the  acquisition  life-cycle.  Hie 
literature  review,  however,  pointed  out  that  the  importance  of 
acquisition  strategy  planning  lay  in  the  early  phases  of  the  acquisition 
life-cycle.  Indeed,  the  regulation  concerning  ASPs  stated  that  the 
ASP’s  importance  was  to  bring  expertise  to  bear  early-on  in  a  program. 
Many  of  the  production  phase  program  managers  felt  that  their  strategies 
were  so  fixed  by  that  point  that  the  benefit  of  the  ASP  was  marginal  at 
best. 

ASP  Preparation.  The  findings  showed  that  preparation  for  the  ASP 
was,  perhaps,  the  biggest  detriment  to  adding  positive  value  for  the 
program  manager.  The  average  estimated  effort  required  to  prepare  for 
an  ASP  was  just  over  nine  man-months.  The  majority  of  respondents  did 
not  feel  that  preparation  was  straightforward.  The  opinions  varied  from 
product  division  to  product  division,  suggesting  that  some  product 
divisions  may  have  provided  better  preparation  assistance  than  others. 

In  fairness  to  AFSC  and  the  product  division  staffs,  much  has  been 
done  to  assist  program  managers  in  preparing  for  the  ASP.  However,  some 
of  the  tools  devised  had  greater  impact  than  others.  Indeed,  some  of 
the  tools  that  were  considered  effective  were  not  readily  available  to 
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the  program  managers.  The  research  showed  that  the  product  division 
secretariats  had  the  highest  utilization  rate  and  the  second  highest 
rating  in  terms  of  helpfulness  to  the  program  manager. 

The  video  tape,  on  the  other  hand,  was  considered  to  be  helpful  by 
those  who  used  it,  but  it  was  used  in  only  30%  of  the  cases  studied. 

One  program  manager  who  participated  in  the  filming  of  the  tape, 
commented  that  it  was  really  just  a  "commercial"  for  the  ASP  process  and 
that  it  did  not  really  give  the  PM  any  solid  information  that  would  help 
in  preparing  for  an  ASP.  In  this  vein,  many  of  the  program  managers 
stated  that  the  most  effective  way  to  prepare  for  an  ASP  was  to  either 
sit  in  on  one  or  to  talk  to  a  program  manager  who  had  recently  been 
through  one.  At  some  of  the  smaller  product  divisions,  this  could  be 
very  difficult  to  do  because  of  the  small  number  of  programs  going 
through  the  ASP  process.  The  DSMC  Acquisition  Strategy  Guide  was 
found  helpful  by  those  who  used  it,  but  it  was  used  in  under  40%  of  the 
cases.  AFSC  publishes  lessons  learned  every  six  months  which  highlight 
some  of  the  better  strategies  used  by  program  managers.  However,  the 
current  guidance  emanating  from  AFSC  tends  to  concentrate  more  on  the 
ASP  briefing  than  the  actual  development  of  acquisition  strategies. 

Many  of  the  comments  received  referred  to  a  lack  of  preparation  on 
the  part  of  some  of  the  panel  members.  The  effect  of  this  lack  of 
preparation  was  to  waste  a  lot  of  the  ASP  briefing  discussing  specific 
programmatic  details  rather  then  acquisition  strategy  issues.  This 
author  attended  one  joint  ASP  and  observed  this  problem  first-hand.  A 
large  part  of  the  problem  has  been  due  to  the  headquarter’s  declining 
role  in  the  management  of  major  programs  and,  thus,  a  resultant  lack  of 
communication  with  the  program  offices.  This  translated  into  the  panel 
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members  having  to  be  educated  on  the  specific  programs  during  the  ASPs 
themselves . 

Conduct  of  the  ASP.  The  findings  to  questions  7  and  17  showed 
that  the  ASP  was  being  conducted  professionally  and  in  a  non- 
confrontational  manner.  Through  the  comments  of  the  program  managers, 
it  appears  that  the  non-confrontational  aspect  of  the  ASP  is  important 
to  its  continued  success.  Many  commented  that  the  ASP  should  be  less 
formal  and  that  it  should  take  on  more  of  a  dialogue  type  approach 
rather  then  the  current  briefing  approach.  And  in  discussing 
confrontation  as  it  applies  to  the  ASP,  it  should  be  noted  that  the 
demeanor  of  the  participants  is  what  is  important,  not  the  avoidance  of 
controversial  program  strategy  issues. 

Responses  to  questions  18  and  19  showed  that  both  the  product 
division  panel  members  and  the  headquarters  AFSC  panel  members  added 
value  to  the  process  and  that  their  respective  values  were  in  different 
areas.  Therefore,  major  program  ASPs  should  continue  to  include  both 
groups. 

The  findings  to  question  10  of  the  interview  guide  showed  that  the 
ASP  minutes  were  effective.  They  were  conveying  the  correct  information 
to  the  ASP  participants,  and  they  were  being  published  in  a  timely 
manner. 

Other  Benefits  and  Problems  with  the  ASP.  The  primary  benefit  of 
the  ASP  was  in  its  use  as  a  "sanity  check"  of  the  program  managers' 
strategies.  Another  perceived  benefit  was  that  it  forced  discipline 
into  the  strategy  formulation  process.  It  also  provided  a  vehicle  for 
the  cross-fertilization  of  lessons  learned  across  the  command.  The 
problems  associated  with  the  ASP  process  included  scheduling 
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difficulties,  the  amount  of  information  required  for  the  ASP  briefings, 
and  the  rigidity  of  the  presentation  format. 

Summary.  The  research  has  shown  that  while  ASP  recommendations 
did  not  change  the  fundamental  aspects  of  program  strategies,  they  did 
alter  minor  aspects  of  program  strategies.  The  majority  of  the 
recommendations  were  being  implemented  and  the  value  to  the  programs  of 
these  implemented  recommendations  was  considered  to  be  positive. 

Overall,  the  ASP  process  was  given  high  marks  by  program  managers  and 
panel  members  alike. 

The  research  did  uncover  some  problems  in  the  process.  Among 
these  were  the  amount  of  preparation  a  program  manager  had  to  engage  in 
prior  to  an  ASP,  a  lack  of  programmatic  information  being  available  to 
panel  members  prior  to  an  ASP,  and  the  large  number  of  programs  already 
in  the  production  phase  of  the  acquisition  life-cycle  which  went  through 
the  ASP  process  and  which  received  marginal  benefit  from  the  process. 

The  next  section  of  this  chapter  discusses  recommendations  for  improving 
the  process. 

ASP  Process  Recommendations 

It  is  clear  from  the  research  that  the  Air  Force  has  a  good  ASP 
process.  In  fact,  one  of  the  respondents  remarked  that  the  "ASP  was  one 
of  the  things  we  were  doing  right"  in  the  acquisition  field. 
Nevertheless,  the  process  can  be  improved.  The  improvements  suggested 
from  the  research  findings  are  described  below. 

1.  Continue  the  ASP  Process.  First,  and  foremost,  the  ASP 
process  should  be  continued.  The  early  perception  that  the  ASP  was  just 
another  bureaucratic  square-filling  exercise  has  been  refuted  by  the 
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findings  of  this  research.  The  research  clearly  shows  that  the  ASP  is 
of  value  to  the  program  manager  and  that  its  long  term  benefits  outweigh 
its  short  term  penalties.  It  should  be  noted,  however,  that  the  ASP’s 
value  varies  depending  on  the  acquisition  phase  of  the  particular 
program  involved.  This  issue  forms  the  basis  for  the  next 
recommendation . 

2.  Delegate  Production  ASPs.  Many  of  the  program  managers  stated 
that  the  value  of  the  ASP  declined  els  the  program  matured.  The 
expertise  of  the  product  divisions  with  the  purchase  and  fielding  of 
particular  types  of  weapon  systems  is  sufficient  to  provide  the 
production  program  managers  with  the  advice  which  they  need.  This  would 
avoid  the  additional  costs  and  effort  required  with  Headquarters  AFSC 
participation  while  achieving  similar  results.  Additionally,  ASPs  for 
follow-on  production  contracts  should  be  discontinued  altogether. 

3.  Integrate  the  ASP  with  the  H30  Structure.  With  Headquarters 
AFSC  role  in  the  management  of  Air  Force  acquisition  programs  on  the 
decline,  the  involvement  of  the  program  executive  officers  (PEOs)  in  the 
ASP  process  becomes  more  critical.  Allowing  the  PEOs  to  co-chair  ASPs 
would  bring  them  into  the  acquisition  strategy  formulation  loop  and 
would  put  ASP  recommendations  on  the  same  level  as  the  programs’ 
decision  making  authority.  The  AFSC  panel  would  then  function  more  as 
an  independent  advisory  board  with  the  PEO  making  the  decisions  on 
whether  the  panel’s  recommendations  should  be  implemented  or  not. 

4.  Insist  on  High-Level  User  Participation.  Some  of  the  program 
managers  and  panel  members  described  ASPs  in  which  the  user 
representatives  had  no  real  authority  to  commit  to  anything  on  the  part 
of  the  using  command.  In  several  cases ,  the  representative  did  not  have 
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an  understanding  of  the  program  being  discussed  or  the  acquisition 
process.  To  adequately  address  requirements  and  risk,  knowledgeable 
representation  on  the  part  of  the  user  is  essential.  Anything  less  will 
result  in  problems  for  the  user  downstream. 

5.  Continue  to  Emphasize  Quality  Panel  Membership.  Several 
respondents  pointed  out  that  the  ASP  was  a  very  personality  driven 
process.  Its  continued  success  hinges  on  the  quality  of  the  personnel 
making  up  the  panels  at  both  the  headquarters  and  product  division 
levels.  "Off-the-wall"  comments,  championing  of  causes,  and  comments 
reflecting  a  lack  of  understanding  for  a  program  all  work  together  to 
reduce  the  credibility  of  the  panel.  Hie  panel's  credibility  makes  the 
difference  in  the  usefulness  of  the  ASP.  Panel  chairpersons  must 
continue  to  monitor  the  performance  of  all  panel  members  and  insist  on 
replacing  panel  members  only  with  other  quality  personnel.  The 
chairpersons  should  insist  that  primary  panel  members  attend  as  many  of 
the  briefings  as  possible. 

6.  Recognize  the  Directive  Nature  of  the  ASP.  Panel  members  need 
to  understand  that,  while  the  ASP  is  advisory  in  nature,  their  positions 
and  authority  serve  to  make  the  ASP  implicitly  directive.  Panel  members 
must  recognize  this  and  avoid  making  comments  which  are  not  well  thought 
out. 

7.  Insist  on  Proper  Preparation  by  Panel  Members.  The  ASP  is  not 
a  cost  effective  vehicle  for  providing  panel  members  with  programmatic 
information.  The  tight  schedules  of  panel  members  which  dictate  the 
length  of  the  ASP  briefings  require  that  they  be  well  versed  in  the 
programmatic  issues  before  the  panel  is  convened.  Every  minute  spent  in 
an  ASP  providing  background  information  takes  away  a  minute  from  the 
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discussion  of  the  program's  important  acquisition  strategy  issues.  The 
result  is  that  some  of  the  issues  which  require  heavy  debate  may  be 
glossed  over  while  others  may  not  be  discussed  at  all.  An  information 
package  describing  program  background  should  be  forwarded  to  the  AFSC 
ASP  secretariat  for  distribution  to  the  panel  members  several  weeks 
prior  to  the  convening  of  an  ASP.  Panel  members  would  then  have  the 
responsibility  to  review  the  package  before  the  ASP  itself.  While  this 
would  increase  the  workload  for  the  program  managers  in  the  short  term, 
it  would  be  expected  to  result  in  a  much  more  useful  dialogue  during  the 
ASP.  The  pre-meetings  of  all  panel  members  to  discuss  the  issues 
expected  to  surface  at  upcoming  ASPs  should  be  formalized. 

8.  Allow  for  Greater  Flexibility  and  Interaction.  While  it  is 
clear  that  the  panel  members  require  certain  information  to  be 
presented,  the  program  manager  should  be  given  the  flexibility  to 
present  the  material  in  a  manner  which  fits  the  program  and  his  or  her 
management  style.  It  should  be  made  clear  that  the  ASP  Presentation 
Guide  is  really  just  a  guide  and  that  strict  adherence  to  its  format  is 
not  required.  Likewise,  in  a  less  formal  atmosphere ,  the  program 
manager  must  assume  the  responsibility  for  getting  the  required 
information  across  to  the  panel  members .  Hie  ASP  regulation  and  the 
presentation  guide  should  be  changed  to  tailor  the  type  of  information 
required  to  the  particular  phase  of  a  program.  The  importance  of  the 
ASP  lies  in  the  cross-flow  of  information  between  the  participants  and, 
therefore,  the  ASP  briefing  format  should  be  subordinate  to  its 
interactive  nature. 

9.  Continue  Efforts  to  Standardize  Risk  Assessment.  Part  of  the 
success  of  the  ASP  hinges  on  its  ability  to  predict  what  will  happen  in 
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the  future.  Key  to  this  is  adequate  risk  assessment  of  the  programs. 
There  is  currently  no  standardized  model  within  the  Air  Force  to  assess 
program  risks.  Each  program  manager  who  briefs  an  ASP  goes  about  the 
risk  assessment  task  in  a  slightly  different  way.  This  makes  it 
difficult  on  the  panel  members  to  impart  the  best  advice  for  a  program. 
The  command  has  embarked  on  an  effort  to  develop  standardized  risk 
assessment  models  and  should  continue  to  do  so. 

10.  Encourage  the  Use  of  the  ASP  an  an  Ad  Hoc  Basis.  The  ASP  is 
currently  required  for  programs  nearing  a  milestone  decision  point. 
However,  programs  are  often  redirected  between  milestones.  This 
redirection  can  come  in  the  form  of  reduced  funding,  changing  threat 
environments,  and  changing  user  requirements,  to  name  a  few.  Each  of 
these  changes  can  have  a  significant  impact  on  program  strategy.  Yet, 
the  ASP  regulation  makes  no  provisions  for  convening  ad  hoc  ASPs  at  the 
request  of  the  program  manager.  Accordingly,  the  regulation  should  be 
changed  to  allow  for  program  manager  requested  ASPs. 

11.  Standardize  ASP  Briefing  Schedules.  A  complaint  of  program 
managers  was  in  the  difficulty  in  scheduling  the  ASP  briefings  to 
conform  to  the  schedules  of  the  high-level  personnel  involved. 
Headquarters  AFSC  and  the  product  divisions  should  set  aside  a  specific 
day  or  days  during  each  month  for  the  express  purpose  of  conducting 
ASPs.  This  would  result  in  fewer  scheduling  difficulties  and  higher 
structure  for  the  preparation  process. 

12.  Encourage  the  Use  of  Video  Tele-Conferencing.  Interestingly, 
the  author  did  not  hear  of  any  instances  in  which  this  method  of 
communication  was  used.  The  standardized  locations  and  format  of  ASP 
briefings,  however,  make  the  ASP  a  prime  candidate  for  the  use  of  video 
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tele-conferencing.  Hie  benefits  of  video  tele-conferencing  include 
reduced  travel  budgets  and  more  efficient  use  of  resources. 

13.  Produce  a  Video  Tape  of  an  Actual  ASP  Briefing.  Hie  video 
tape  produced  by  AFSC  was  considered  by  many  to  be  a  "commercial"  for 
the  ASP  process.  Many  of  the  program  managers  felt  that  sitting  in  on 
ASP  or  talking  to  a  program  manager  who  had  been  through  one  would  have 
been  the  best  way  to  prepare  for  an  ASP.  Using  the  video  tape  mediun  to 
capture  the  essence  of  an  actual  ASP  briefing  followed  by  candid 
comments  from  program  managers  who  had  been  through  the  process,  as  well 
as  panel  members  expressing  their  views  on  what  was  required  in  an  ASP, 
could  be  a  significant  aid  to  a  program  manager  preparing  for  one. 

14.  Publish  an  AFSC  Acquisition  Strategy  Guide.  Hie  DSMC 
Acquisition  Strategy  Guide  was  found  to  be  useful  to  those  program 
managers  who  knew  of  its  existence.  Unfortunately,  many  did  not  know  it 
existed.  AFSC  could  combine  its  semi-annual  lessons  learned  package 
into  an  acquisition  strategy  guide  which  could  be  distributed  across  the 
command.  This  would  have  the  potential  to  reach  many  more  program 
managers  than  the  DSMC  guide.  Providing  assistance  to  the  program 
manager  before  he  or  she  formulated  the  program  strategy  could  be  of 
more  use  than  reviewing  it  after  it  was  already  formulated. 

Summary.  This  section  has  made  some  wide-ranging  reconmendations 
regarding  the  ASP  process.  There  are,  however,  areas  which  justify 
further  research. 

Recommendations  for  Further  Research 

This  thesis  has  concentrated  on  the  joint  ASPs  conducted  by  AFSC 
and  its  product  divisions.  Of  concern  is  how  the  ASP  i-  being 
implemented  at  the  product  division  and  lower  levels.  Several  of  the 
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It  is  hoped  that  the  recommendations  suggested  by  the  findings  of  this 
research  will  be  of  benefit  to  the  personnel  who  acquire  the  Air  Force’s 
weapon  systems. 
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Appendix  A:  Suggested  Topics  for  Discussion  at  an  ASP 

AFSCR  800-53  Attachment  1  19  July  1989  , 

1.  Requirements  and  Program  Description: 

♦Overall  acquisition  strategy 
♦Integrated  program  schedule 
♦Program  assumptions  and  alternatives 
♦Program  margin  and  slack 
♦Could-cost  application 
User  requirements  known 
Requirements  updated 

Disconnects  identified  and  trade-offs  accomplished 

User  involvement 

People  resources 

Potential  competitors 

Lessons  learned  and  experience 

Political  concerns 

Security  issues 

TQM  application 

2 .  Engineering : 

♦Engineering  strategy 
♦Engineering  schedule 

♦Engineering  assumptions  and  alternatives 
♦Engineering  margin  and  slack 
♦Could-cost  application 
Threat  response 

Reliability,  maintainability,  and  producibility 

Technology  transition 

Requirements  allocation 

Control  of  development  process 

Risk  assessment  and  reduction 

Operational  use  considerations 

Software  control 

Industrial  base  activity 

Productivity  enhancement 

Production  operations  requirements 

Environmental  Bafety  and  occupational  safety  issues 

Hardware  and  software  systems  integration  concept  definition 

TQM  application 

3.  Test:  * 

♦Test  strategy 
♦Test  schedule 

♦Test  assumptions  and  alternatives  « 

♦Test  margin  and  slack 

♦Could-cost  application 

Test  and  evaluation 

Developmental  test  and  evaluation 

Operational  test  and  evaluation 

Test  resource  summary 

Transition  from  development  to  production 
TQM  application 


120 
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4.  Support: 

♦Support  strategy 
♦Support  schedule 

♦Support  assumptions  and  alternatives 
♦Support  margin  and  slack 
♦Could-cost  application 

Maintenance  and  support  concept  definition 

Lessons  learned  application 

Life  cycle  cost  management 

Logistics  support  analysis  application 

Resource  development 

Fielding  strategies 

Mission  planning 

Post-production  support 

Software  maintenance  and  support  concept  definition 
TQM  application 

5.  Budget: 

♦Funding  requirements  and  budget 

♦Budget  assumptions  and  alternatives 

♦Budget  margin  and  slack 

Legislation 

Unfunded  requirements 

Disconnects 

Planning,  programming,  and  budgeting  system 
Independent  cost  analysis 
TQM  application 

6.  Business: 

♦Business  strategy 
♦Business  schedule 

♦Business  assumptions  and  alternatives 

♦Business  margin  and  slack 

♦Could-cost  application 

Contract  type,  parameters,  and  incentives 

Competition 

Long  lead 

Letter  contract  and  unpriced  order 
Should-cost 

Design-to-cost  and  life  cycle  cost 
Multiyear 

Small  and  disadvantaged  business  opportunities 
Warranty 

Government-furnished  property,  equipment,  and  data 

Value  engineering 

Indemnification 

Allied  or  foreign  participation 
Foreign  military  sales 
Unique  clauses  and  special  provisions 
TQM  application 

♦  Indicates  a  core  topic  which  describes  the  essential  strategy  of  each 
section. 
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Appendix  B:  Interview  Guide 


INTERVIEW  # 


INTEJODUCTION 

I  am  conducting  this  interview  as  part  of  a  Masters  Degree  thesis 
sponsored  by  the  Air  Force  Institute  of  Technology.  The  objective  of 
the  thesis  is  to  determine  if  the  Acquisition  Strategy  Panel  (ASP) 
process  is  of  value  to  the  program  manager  in  the  development  and 
implementation  of  program  strategy.  Hie  end  result  of  this  thesis 
research  is  expected  to  be  a  recommendation  for  continuance, 
discontinuance,  or  modification  of  the  ASP  process.  To  do  so,  I  am 
conducting  interviews  with  program  managers  and  other  personnel  at  the 
SPO,  product  division  and  headquarters  level  who  have  had  direct 
experience  with  the  process.  To  provide  a  valid  recommendation ,  your 
candor  in  responding  to  the  interview  questions  is  vital. 

Some  of  the  interview  questions  seek  quantifiable  data,  while 
other  seek  subjective  data.  Hie  subjective  questions  will  be  asked  in 
two  parts.  First,  you  will  be  given  a  statement,  and  asked  if  you 
strongly  disagree,  have  no  opinion,  or  strongly  agree  or  are  somewhere 
between  these.  Then  you  will  be  asked  for  specific  comments  regarding 
the  statement .  At  the  end  of  the  interview,  please  feel  free  to  provide 
information  which  you  think  is  important  to  the  ASP  process,  but  was  not 
questioned  earlier. 

I  will  ask  you  several  personal  identification  questions  which  I 
will  use  to  contact  you  should  I  need  clarification  of  your  responses. 
This  information  will  be  kept  separate  from  the  response  sheets  and  will 
be  destroyed  upon  publication  of  the  thesis.  While  portions  of  your 
responses  may  be  used  in  the  thesis,  they  will  not  be  directly 
attributed  to  you. 

At  the  end  of  the  interview,  I  will  leave  a  supplemental  response 
sheet  with  you.  Should  you  think  of  anything  that  might  be  important  in 
characterizing  the  ASP  process  which  we  did  not  cover  in  the  interview, 
please  fill  in  the  supplemental  sheet  and  mail  it  to  me. 

THANK  YOU  FOR  YOUR  TIME! 


SCOTT  C.  HARDIMAN,  Capt,  USAF 
Student,  School  of  Systems  and  Logistics 
Air  Force  Institute  of  Technology 
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INTERVIEW  GUIDE 


INTERVIEW  # 


DEMOGRAPHIC  DATA 

1.  Which  of  the  following  best  describes  your  role  in  the  ASP  process? 

A.  Program  Manager 

B.  Contracting  Officer 

C.  Product  Division  ASP  Focal  Point 

D.  Product  Division  ASP  panel  member 

E.  Command  ASP  panel  member 

F.  Other.  Specify: 


2.  Are  you  currently  involved  in  the  ASP  process?  Yes  No.  If 

not,  how  long  ago  were  you  involved?  _ 

If  your  association  is  with  a  specific  program,  when  was  the  ASP 

conducted?  _ .  At  what  point 

in  the  program’s  lifecycle  did  the  ASP  occur?  _ 


3.  If  you  were  involved  at  the  SPO  level,  what  was  the  value  of  the 
program? 

$ _ R  &  D  $ _ Production 

4.  The  specific  ASP(s)  that  you  have  been  involved  with  was  (were)  a 

_  Joint  ASP  _ Product  Division  ASP 

(AFSC  &  Prod  Div) 

YOU  WILL  NOW  BE  ASKED  SEVERAL  OPINION  QUESTIONS.  PLEASE  USE  THE 
FOLLOWING  SCALE  IN  RESPONDING  TO  THESE  QUESTIONS. 

1  2  3  4  5 

STRONGLY  MODERATELY  NO  OPINION  MODERATELY  STRONGLY 

DISAGREE  DISAGREE  AGREE  AGREE 

The  overall  ASP  process  is  or  was  of  value  to  the  program  manager. 

1  2  3  4  5 

ASP  PREPARATION  AND  GUIDANCE 


1.  Preparation  for  the  ASP  was  straightforward. 

1  2  3  4  5 
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INTERVIEW  # 


2.  The  following  items  were  helpfull  in  preparing  for  the  ASP  (0  for 
not  used,  N  for  unaware  of  its  existence). 


AFSC  Regulations 
Product  Division  Regs 
Product  Division  Handbook 
AFSC  Video  Tape 
AFSC  ASP  Focal  Point 
Prod  Div  ASP  Focal  Point 
DSMC  Acq  Strategy  Guide 
ACSC  Acq  Strategy  Guide 


3.  Were  there  other  items  that  were  not  listed  above  which  served  as  an 

aid  in  preparing  for  the  ASP?  _ Yes  _ No.  If  yes,  what 

were  they? _ _ ___ 


♦ 


4.  Please  estimate  the  amount  of  effort  expended  in  preparing  for  the 
ASP. 


#  of  personnel  _ .  Manhours  _ .  Overall  length  of  time  _ 

Are  these  estimated  or  actual  figures?  _  Est  _  Act 

5.  Preparation  for  the  ASP  resulted  in  an  altered  program  strategy. 

STRONGLY  DISAGREE  STRONGLY  AGREE 

1  2  3  4  5 

If  so,  how? 


6.  Do  you  have  any  other  comments  on  ASP  preparation  or  suggestions  for 
changing  the  preparation  process? 


THE  ACQUISITION  STRATEGY  PANEL 

7.  The  ASP  was  conducted  in  a  non-confrontational  way. 
1  2  3  4  5 

Specific  comments: 
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INTERVIEW  # 

8.  Comments  received  from  panel  members  were  constructive. 

STRONGLY  DISAGREE  STRONGLY  A®EE 

1  2  3  4  5 

Specific  Comments: _ 


9.  The  ASP  redirected  overall  program  strategy. 

1  2  3  4  5 


If  so,  how? 


10.  Written  minutes  of  the  ASP  briefing  paralleled  oral  coranents  made 
during  the  briefing. 

1  2  3  4  5 

If  not,  how  did  they  differ  and  what  was  the  source  of  the 
difference? 


11.  Direction  received  from  the  ASP  has  been  implemented. 

1  2  3  4  5 

If  not,  why  not? 


12.  If  ASP  direction  was  implemented,  some  or  all  had  a  positive  impact 
on  the  program. 

1  2  3  4  5 

If  so,  what  was  or  were  the  positive  impacts? 


13.  If  ASP  direction  was  implemented,  some  or  all  had  a  negative  impact 
on  the  program. 

1  2  3  4  5 
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If  so,  what  was  or  were  the  negative  impacts? 


INTERVIEW  # 


QUANTITATIVE  BENEFITS  OR  POSTS  RESULTING  FROM  TOE  ASP 

14a.  Program  Budget:  $ _ 

14b.  Program  Schedule:  _ Hrs 

14c.  Technical  Performance:  _ 

14d.  Program  Quality:  _ 

14e.  Other:  _ 

15.  The  ASP  process  served  to  build  a  consensus  of  support  for  the 
program. 

STRONGLY  DISAGREE  STRONGLY  AGREE 

1  2  3  4  5 

15a.  If  a  consensus  of  support  was  built,  this  served  to  reduce 
oversight  or  program  redirection  later  in  the  acquisition  lifecycle. 

1  2  3  4  5 

Do  you  have  any  comments  on  the  consensus  building  aspect  of  the 

ASP? 


16.  The  basxc  program  strategy  was  developed  prior  to  the  assignment  of 
the  program  manager. 

1  2  3  4  5 

Specific  comments : 


IMPRESSIONS 

17.  The  ASP  was  conducted  in  a  professional  manner  by  all  participants . 
1  2  3  4  5 
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Specific  comments: 


INTERVIEW  # 


18.  The  recommendations  of  the  product  division  panel  members  were/are 
of  value  to  the  program  manager. 

STRONGLY  DISAGREE  STRONGLY  AGREE 

1  2  3  4  5 

Specific  comments: 


19.  The  recommendations  of  AESC  panel  members  were/are  of  value  to  the 
program  manager. 

1  2  3  4  5 

Specific  comments: 


20.  Rate  your  overall  impression  of  the  total  ASP  process,  with  1  being 
lowest  and  5  being  highest. 

1  2  3  4  5 

21.  The  greatest  benefit  of  the  ASP  is: 


22.  The  greatest  problem  with  the  ASP  is: 


23.  The  ASP  could  be  improved  by: 


OTHER  COt-MENTS 
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SUPPLEMENTAL  ASP  INFORMATION  INTERVIEW  # _ 

If  you  would  like  to  embellish  on  your  responses  to  this 
interview,  or  add  information  which  you  do  not  feel  was  covered  in  the 
interview,  please  note  your  comments  below  and  mail  them  to 

AFIT/LSG 

Wright-Patterson  AFB,  OH  45433 
Attn:  Capt  Scott  Hardiman 

Mail  your  comments  so  that  they  will  arrive  by  30  June  1990.  If  you 
have  any  questions,  please  call  me  in  care  of  the  institute  at  Autovon 
785-8989  or  commercial  (513)  255-8989. 

Thanks  again  for  your  help! 


« 
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;ndix  C:  Interview  List 


The  following  is  a  list  of  Air  Force  acquisition  programs  which 
have  been  through  a  Joint  or  Headquarters  AFSC  ASP  and  their  responsible 
program  offices.  In  ALL  cases,  an  attempt  was  made  to  contact  the 
appropriate  person  for  an  interview.  Unfortunately,  the  high  turnover 
inherent  in  the  Air  Force  resulted  in  many  of  the  programs  not  being 
included  in  this  research.  An  asterisk  in  the  "Interviewed"  column 
indicates  that  a  person  from  that  office  was  interviewed  for  this 
thesis.  Programs  having  been  through  more  then  one  ASP  are  only  listed 
once. 


Many  of  the  personnel  interviewed  stated  that  the  greatest  aid  to 
preparing  for  an  ASP  was  to  talk  to  someone  who  had  been  through  the 
process.  It  is  hoped  that  this  list  will  assist  personnel  expecting  to 
participate  in  an  ASP  with  finding  experienced  personnel  to  talk  to. 

Program  Program  Office  Interviewed 


ALS  R/T  SSD/CLH 

MMW  MSD/XR-2 

DSCS-III  SSD/CWH 

IR  Maverick  ASD/SDM  * 

IPE  ASD/YZ  * 

F-16  Multiyear  ASD/YP  * 

Have  Nap  MSD/YGP 

Titan  IV  WSMC/ST 

Seek  Spinner  ASD/VB 

AWACS  RSIP  ESD/TC-1  * 

JTIDS  ESD/TCD  * 

C-17  ASD/YC  * 

ELVUS  SSD/CLU 

Sensor  Fused  Weapon  MSD/YB 

Delta  III  SSD/CLVM 

Combat  Edge  HSD/YA  * 

Alaska  ACMI  MSD/YIC 

NASP  ASD/NA  * 

TITS  ASD/YT  * 

BSTS  SSD/CNB  * 

NAVSTAR  Replenishment  SSD/CWNZ  * 

ASTT  ESD/AT 

Space  Launch  Complex-7  WSMC/ST 

AMRAAM  MSD/YM  * 

C-135  Avionics  Upgrade  ASD/SD  * 

O&M  Contract  AEDC/FK 

OA-IO  ASD/SDF  * 

Brilliant  Pebbles  SSD/CNIW  * 

AGM-130  MSD/YGM 

DSP  Satellite  SSD/CND 

MX  Rail  Garrison  BSD/CV  * 

Space  Based  Radar  SSD/XRS  * 

GPS  MSD/YIB  * 

Joint-STARS  ESD/JS  * 
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The  following  list  identifies  the  Headquarters  AFSC  offices  whose 
personnel  are  members  of  the  command’s  Acquisition  Strategy  Panel.  An 
asterisk  in  the  "Interviewed"  column  indicates  that  a  person  from  that 
office  was  interviewed  for  this  thesis. 

Function  Office  Symbol  Interviewed 


Contracting 

AFSC/PK 

* 

Legal 

AFSC/JA 

* 

Comptroller 

AFSC/AC 

* 

Technology 

AFSC/XT 

* 

Test 

AFSC /DR 

* 

Systems 

AFSC/IG 

Manuf ac turing 

AFSC/PM 

Engineering 

AFSC/EN 

* 

Requirements 

AFSC/EN 

* 

Projects 

AFSC/PKP 

* 

« 


4 
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Appendix  D:  Survey  Results 


The  following  is  the  data  from  the  Quattro  spreadsheet  which  was  used 
in  the  analysis  of  this  research.  Each  column  represents  a  question  asked  in 
the  interview  guide.  Each  row  represents  the  responses  during  a  particular 
interview.  Some  demographic  information  has  been  left  off  to  guarantee  the 
anonymity  of  the  personnel  interviewed. 


ASP  INTERVIEWS 

DEMOGRAPHICS  PREPARATION 

QUESTION  #  JOB  TIME  RM)$  PRD$  PHASE  1  2A  2B 

3 

3 

3 

3 

1 

3 

1 

1 


2C 


2D 


0.5 

0.55 

4 

4 

4 

4 

4 

7 

1 

5 

4 

1 

6 

6 

6 

6 

1.2 

11.5 

4 

3 

6 

4 

7 

6 

0.5 

1 

4 

5 

6 

4 

6 

6 

0.5 

0.5 

1 

2 

3 

4 

4 

4 

1.8 

1.5 

3 

1 

6 

6 

6 

6 

1.2 

3 

4 

4 

6 

6 

6 

1 

0 

2 

4 

4 

3 

3 

7 

1.2 

0.1 

4 

1 

7 

7 

7 

7 

1 

3 

5 

4 

4 

4 

1 

2 

4 

4 

4 

5 

3 

2 

4 

2 

2 

5 

0.2 

0.5 

3 

2 

4 

2 

4 

3 

2 

3 

4 

2 

0.7 

2 

5 

4 

7 

7 

7 

2 

0.1 

0.9 

4 

2 

4 

3 

7 

7 

0.5 

0.7 

3 

2 

2 

2 

7 

7 

0.8 

0.4 

3.1 

3 

2 

4 

3 

4 

4 

0.6 

0.2 

4 

1 

3 

7 

2 

6 

0.8 

0.7 

4 

4 

4 

3 

6 

6 

0.2 

0.01 

0.01 

3 

3 

3 

2 

7 

7 

AGGREGATE 

RESPONSE 

0  29 

9 

10 

18 

21 

20 

20 

20 

20 

AVERAGE 

1.41 

0.34 

2.45 

2 

.71 

MINIMUM 

0.2 

0.00 

0.01 

1 

MAXIMUM 

3 

0.7 

11.5 

5 

STD  DEV 

0.93 

0.27 

3.35 

1 

.28 

VARIANCE 

0.87 

0.07 

11.2 

1 

.63 
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ASP  INTERVIEW 


ASP 


QUESTION  #  2E  2F  20  2H 
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Appendix  E:  Test  for  Reliability  of  the  Instrument 


< 


t 


The  results  of  questions  20  and  0  were  compared  using  a  paired  t- 
test  to  check  for  the  reliability  of  the  instrument.  The  null 
hypothesis,  Ho,  was  that  the  mean  of  the  differences  in  the  pairs  of 
answers  was  0,  or  D  =  0.  The  alternate  hypothesis,  Ha,  was  that  the 
mean  of  the  differences  was  not  equal  to  0,  and  thus  the  responses  had 
different  distributions. 

FREQUENCY  DISTRIBUTION  OF  DIFFERENCES  IN  RESPONSE  TO  QUESTIONS  20  &  0 


VALUE 

N 

-2 

1 

!* 

-1 

2 

!** 

0 

14 

J  ************** 

1 
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;********* 
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1 

!  * 

27  Cases  Plotted. 


CHECK  FOR  NORMALCY  OF  THE  DIFFERENCES 


RANKITS 
3.0  + 


1.0  + 
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:  4 

:  4 

:  3 

-1.0  +  3 

;  2 

:  + 


-3.0  + 

-+ - + - + - 

DIF  -2.0  -1.0  0.0 

APPROX.  WILK-SHAPIRD  0.8550 


+ 

+  + 

3 

3 

+ 

The  Wilke-Shapiro 
coefficient  checks 
for  normalcy  of 
the  distribution. 

The  closer  the 
coefficient  is  to 
1 ,  the  more  normal 

-+ - +-  is  the  dist. 

1.0  2.0 

27  CASES  PLOTTED 


PAIRED  T  TEST  FOR  QUESTION  0  RESPONSE  -  QUESTION  20  RESPONSE 

MEAN  2.778E-01  STD  ERROR  1.609E-01 
T  1.73 

DF  26 

For  a  two-sided  paired  t-test  with  a  95%  confidence  level  and  26  degrees 
of  freedom,  the  critical  t  value  is  2.056.  Ho  is  rejected  if  T  2.056 
or  T  -2.056.  The  observed  t  value,  1.73,  does  not  fall  into  the 
rejection  region  and  the  null  hypothesis  is  accepted.  Thus,  with  a  95% 
confidence,  the  distributions  of  questions  20  and  0  are  considered  the 
same  and  the  instrument  is  considered  reliable. 
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Appendix  F:  Test  for  Construct  Validity  of  the  Instrument 

The  following  tables  represent  the  cross-tabulations  of  the 
responses  to  questions  8,  12,  and  13  with  the  responses  to  question  20. 
The  section  entitled  Validity.  Reliability  and  Practicality  of  the 
Instrument  in  Chapter  III  discussed  the  criteria  for  determining  the 
construct  validity  of  the  instrtment.  Chapter  IV  discussed  the  results 
of  the  cross-tabulations  depicted  below. 

Table  2.  Cross-Tabulation  of  Question  8  With  Qjestion  20 
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Table  3.  Cross-Tabulation  of  Question  12  With  Question  20 
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Table  4.  Cross-Tabulation  of  Question  13  With  Question  20 
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